Hello to Spark World

For more updates check out my Apache Spark exploration @ https://github.com/Mageswaran1989/aja/

Spark Memory and Task Calculation

For latest updates check out my Apache Spark exploration @ https://github.com/Mageswaran1989/aja/


How many tasks can I run in my cluster?

M(Machines) * E(Executors/Machine) * C(Cores per Executor / 1 core for each task) = T(Tasks)

Eg: 12 * 2 * 12 = 288 tasks

How much amount of RAM is under usage?

0.9 (spark.storage.safetyFraction) * 0.6 (spark.storage.memoryFraction) * M (Machines) * E (Executors/Machine) * S (Size in GBs per executor)

Eg: 0.9 * 0.6 * 12 * 2 * 26 = 336.96 GB



Reference: http://0x0fff.com/spark-architecture/

Enhancing Blogspot with 3rd party Scripts

1. Follow below links to enable LaTeX support in your blog.
http://www.codecogs.com/latex/integration/blogger/install.php
http://tex.stackexchange.com/questions/13865/how-to-use-latex-on-blogspot

2. Follow below link to integrate Git in your blog.
http://robertgreiner.com/2012/04/using-github-as-a-syntax-highlighter/