Networking Dashboard v01

Networking Dashboard is a tool that shows internals of networking of Firefox. It's an add-on that I have made as part of GSoC project and can be used by typing 'about:networking-dashboard' in URL bar after installing this add-on. It shows current open connections with host names, how many of them are active/idle, if they are using SPDY or not, if they are using SSL or not, DNS cache entries, total data sent and received, data related to WebSocket connections etc. I have blogged about it in detail in my previous post.
Following are some screen shots which might give you some idea about the tool.
Timeline shows graphical view of total data sent and received after starting the add-on.
One feature that I couldn't implement as part of add-on is header window which shows sent and received headers of the packets Http connections.

I have used jQuery and flot - a pure Javascript plotting library for jQuery -  for building the add-on. I can make the add-on available but it won't work because back end part of the code hasn't been migrated to mozilla-central yet.

Http Connections

Socket Connections

DNS cache entries

WebSocket connections

Timeline showing graph of total data sent and received

GSoC update

So far I have finally finished implementing back-end interfaces- nsIHttpConnectionLog.idl, nsiWebSocketConnectionLog.idl, nsISocketLog.idl and nsIDNSEntries.idl. for Networking Dashboard.
Now, only thing remains is a UI for the add-on using these interfaces. It's very important part of the project since it will make sense of the json strings returned by these interfaces. Without a UI they are merely strings of data. I will probably be using jquery UI for this. It will be a tabular UI, a tab per interface/feature. Implementation of UI will conclude my GSoC project.

Update of GSoC Till Midterm Evaluation

I have been working on Networking Dashboard for quite a while now and I think the back end part of the project is nearly done. It involves porting out information related to connections of different protocols(HTTP,WebSocket,etc.) through different interfaces.
This blog is an update on how much has been done so far and what am I going to get done in future.

What has been done so far?
So far I have implemented four interfaces. Every interface returns a JSON string containing different information in different fields.

1) nsIHttpConnectionLog.idl :- This interface returns following information related to current HTTP connections in form of a JSON string,

- host : Host name of the HTTP connection
- port : port no. of the connection
- spdy : If connection using spdy or not
- active : rtt(Round Trip Time) and ttl(Time To Live) of active connections in respective fields
- idle : rtt and ttl of idle connections in respective fields


The connection data has been collected from nsHttpConnectionMgr.h where data related to http connection is maintained in form of a hash table.
Following is the example of a JSON string which is returned from this interface after opening www.google.com followed by www.twitter.com:

 {  
   "host" : ["mail.google.com","www.google-analytics.com","accounts.google.com","api.twitter.com","ocsp.thawte.com","localhost","accounts.youtube.com","a0.twimg.com","www.mozilla.org","ssl.gstatic.com","ssl.google-analytics.com","gmail.com","scribe.twitter.com","twitter.com","mail.google.com"],  
   "port" : [443,80,443,80,80,80,443,80,80,443,443,80,80,80,80],  
   "active" : [{  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [143]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   }],  
   "idle" : [{  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [95]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [79]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0,0,0,0],  
     "ttl" : [89,86,85,85]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [85]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   }],  
   "spdy" : [true,false,true,false,false,false,true,false,false,true,true,false,false,false,false]  
 }  

Empty ttl and rtt array means that that there is no connection with the host.
You can have a better look at it using any online JSON editor. Sorry for posting such long JSON. It looks ugly if I don't format it this way.


2) nsIWebSocketConnectionLog.idl :- This interface returns following information related to current WebSocket connections in form of a JSON string,

- hostport : Host name and port of the WebSocket connection
- msgsent : No. of messages sent
- msgreceived : No. of messages received
- encrypted : If the connection secure or not

There is no maintained data available for WebSocket. So all the data related to the connections is maintained at the implementation of the interface(in file nsWebSocketConnectionLog.h) Every time a new connection is made with new host or an old connection is closed, an entry is made or removed from the database if an instance of the interface has been created.

I found websites using WebSocket from www.websocket.org.
Here is one example of JSON string returned from the interface after opening http://rumpetroll.com/ and wandering around a little bit and after that http://www.kaazing.me :

 {  
   "hostport" : ["rumpetroll.motherfrog.com:8180","demo.kaazing.com"],  
   "msgsent" : [76,8],  
   "msgreceived" : [99,223],  
   "encrypted" : [false,true]  
 }  

3) nsISocketLog.idl - This interface returns following information related to current Socket connections in form of a JSON string,

- host : peer host ip
- port : peer host port
- tcp : if the connection is TCP or not

Data related to socket is collected from nsSocketTransportService2.h.
The data returned after opening www.google.com and http://twitter.com/
in above example is:

 {  
   "host" : ["74.125.236.102","199.59.149.243","124.124.201.161","199.59.150.41","124.124.201.161","124.124.201.161","124.124.201.161","199.59.150.7","74.125.236.98","74.125.236.117","173.194.36.30","74.125.236.111","209.85.175.84","74.125.236.118","199.7.51.72","74.125.236.117"],  
   "port" : [20480,20480,20480,20480,20480,20480,20480,20480,47873,47873,47873,47873,47873,20480,20480,20480],  
   "tcp" : [true,true,true,true,true,true,true,true,false,false,false,false,false,true,true,true]  
 }  

This data is not much I am hoping to add some more fields(like total data sent and received) to this interface.
4) nsDNSEntries.idl - This interface returns cached entries in DNS cache with  
    following fields,

- hostname : name of the host
- hostaddr : ip address of the host
- family : ip address family(ipv4 or ipv6)
- expiration : time when the cache entry expires(in minutes)

Well, this interface is near completion(believe me,it is). One little difficulty is stopping me from posting it's output here. May be I post it in the comment after it's done. It's going to contain above fields.

This concludes what I have completed so far.
I have been maintaining a mozilla-central repository at bitbucket.org and committing to my updates to branch 'Networking Dashboard'. You can have a look at it if you want. To test these interfaces, i have made a temporary addon using jetpack without any GUI.

What am I going to do next?
First of all I am going to finish the DNS interface.
Since basic interfaces have been implemented, I can now start working on GUI part of the addon. On the way of doing that, I might need to make differences to the interfaces or add some new data.
I also want to include time stamps, size of data sent/received to above interfaces of different protocols. These two feature are very important to the addon. I will work on them after I am done giving a face to the 'Networking Dashboard' addon.

This was an overview of my GSoC work so far. I have learnt a lot of new things about networking and Mozilla internals throughout this period. Thanks for reading. I hope you download my addon in future :)

Update on GSoC (May 21 - May 13)

I have ported out connection data related to protocols like HTTP,WebSocket and also related to sockets to javascript.

Now I data related to these protocols can be accessed through new interfaces.

Data related to different protocol in particular are -

HTTP -            Host name,  Port no., No. of active connections, No. of idle
                        connections, if connection is using SPDY or not (In future - socket
                        identifiers for particular connections, RTT for connections, TTL etc.)

WebSocket -   Host name, Port no. (In future - No of messages sent so far)

Socket -          Socket identifier and packets sent or received via that socket can be 

                        seen. (Not perfect right now.)

We can search for different HTTP connections for their packets in Socket data via identifiers.

Next, I might add a new interface over all of these interface which combines all of these data and returns a single JSON string. We can also do some calculations using these data.

Update on GSoC (May 7 - May 20)

Though the coding period starts from 21st May, I have started working on new IDL as part of my project.

I started working from 7th May. My first try is to get Http connection data like- hostname, port, No. of active connections, No. of idle connections. In first week I was getting my head around how to create a new idl, xpcom module, etc. Later on I came to know that I don't have to create new module, I can add new idl to Network module. So, now I have created my new idl and implemented it in c++. Only thing lest is to test it and see if I am getting Http connection data in a javascript or not. But it's not that easy. It seams that firefox doesn't allow use of UniversalXPConnect in a normal javascript. So now I have two options, either use jsctypes(which will take me time to understand) or create an add on to access the idl. I am going with the latter. Right now I am facing some difficulties in developing new add on with a nightly build but I will get over it.

GSoC 2012 - Networking Dashboard

My proposal for 'Networking Dashboard' for 'Mozilla' got selected for GSoC 2012 today :).
I have been a contributor to Mozilla for a long time now. Till now mostly it was bug fixes.
But now I will be working on something real for Mozilla. My project is to make a tool for developers which allows them to have an insight of Mozilla networking. Google chrome already has something similar - "chrome://net-internals". This tool will be implemented as an add-on. The link to my proposal is here. It has a description about what functionality will be implemented. I wish I have a link to the add-on soon. I might blog about my experience with necko and about my progress in future. Stay tuned.