Using Interactive JavaScript for Data Visualization in Hadoop on Windows Azure Cluster

Apache Hadoop on Windows Azure is integrated with a web-based interactive JavaScript console, which allows you to perform

  • Perform HDFS operations, including uploading/reading files to/from the HDFS
  • Run MapReduce programs from .js scripts or JAR files, and monitor their progress
  • Run a Pig job specified using a fluent query syntax in JavaScript, and monitor its progress
  • Visualize data with graphs built using HTML5

Once you have built your Hadoop cluster on Windows Azure, you can launch Interactive JavaScript web console directly as below:

After the Interactive JavaScript is launched you can enter a set of commands or your own JavaScript directly from the browser:

For our example we will do the following:

  1. Upload a data file to HDFS
  2. Run JavaScript to parse the data
  3. Run Graph Visualizer with parsed data

Let’s collect some real time data about mobile market share for 2011:

http://www.netmarketshare.com/operating-system-market-share.aspx?qprid=9&qpcustomb=1

Create a txt file locally as mobilesharedata.txt locally:

iOS 52.10
JavaME 21.27
Android 16.29
Symbian 5.76
BlackBerry 3.51
Other 1.07

Upload this file to HDFS from Interactive JavaScript console:

js> fs.put()
File uploaded.

Verify the file is uploaded:

js> #ls
Found 2 items
drwxr-xr-x – avkash supergroup 0 2012-01-02 20:37 /user/avkash/.oink
-rw-r–r– 3 avkash supergroup 81 2012-01-02 20:52 /user/avkash/mobilesharedata.txt

Let’s read the data this

js> file = fs.read(“/user/avkash/mobilesharedata.txt”)
iOS 52.10
JavaME 21.27
Android 16.29
Symbian 5.76
BlackBerry 3.51
Other 1.07

Now parse the data as below:

js> data = parse(file.data, “OS, MarketShare:long”)
[
0: {
OS: “iOS”
MarketShare: 52
}
1: {
OS: “JavaME”
MarketShare: 21
}
2: {
OS: “Android”
MarketShare: 16
}
3: {
OS: “Symbian”
MarketShare: 5
}
4: {
OS: “BlackBerry”
MarketShare: 3
}
5: {
OS: “Other”
MarketShare: 1
}
]

Set the graph visualization settings as below:

js> options = { title: “Mobile OS Market Share in 2011”, orientation: 20, x: “OS”, y: “MarketShare” }
{
title: “Mobile OS Market Share in 2011”
orientation: 20
x: “OS”
y: “MarketShare”
}

Now let’s visualize the data in Bar Graph:

 
js> graph.bar(data, options)

Now let’s visualize the data in Pie Graph:

js> graph.pie(data, options)

Keywords: Windows Azure, Hadoop, Apache, BigData, Cloud, MapReduce

Advertisements

2 thoughts on “Using Interactive JavaScript for Data Visualization in Hadoop on Windows Azure Cluster

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s