Update of GSoC Till Midterm Evaluation

I have been working on Networking Dashboard for quite a while now and I think the back end part of the project is nearly done. It involves porting out information related to connections of different protocols(HTTP,WebSocket,etc.) through different interfaces.
This blog is an update on how much has been done so far and what am I going to get done in future.

What has been done so far?
So far I have implemented four interfaces. Every interface returns a JSON string containing different information in different fields.

1) nsIHttpConnectionLog.idl :- This interface returns following information related to current HTTP connections in form of a JSON string,

- host : Host name of the HTTP connection
- port : port no. of the connection
- spdy : If connection using spdy or not
- active : rtt(Round Trip Time) and ttl(Time To Live) of active connections in respective fields
- idle : rtt and ttl of idle connections in respective fields


The connection data has been collected from nsHttpConnectionMgr.h where data related to http connection is maintained in form of a hash table.
Following is the example of a JSON string which is returned from this interface after opening www.google.com followed by www.twitter.com:

 {  
   "host" : ["mail.google.com","www.google-analytics.com","accounts.google.com","api.twitter.com","ocsp.thawte.com","localhost","accounts.youtube.com","a0.twimg.com","www.mozilla.org","ssl.gstatic.com","ssl.google-analytics.com","gmail.com","scribe.twitter.com","twitter.com","mail.google.com"],  
   "port" : [443,80,443,80,80,80,443,80,80,443,443,80,80,80,80],  
   "active" : [{  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [143]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   }],  
   "idle" : [{  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [95]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [79]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0,0,0,0],  
     "ttl" : [89,86,85,85]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [85]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   }],  
   "spdy" : [true,false,true,false,false,false,true,false,false,true,true,false,false,false,false]  
 }  

Empty ttl and rtt array means that that there is no connection with the host.
You can have a better look at it using any online JSON editor. Sorry for posting such long JSON. It looks ugly if I don't format it this way.


2) nsIWebSocketConnectionLog.idl :- This interface returns following information related to current WebSocket connections in form of a JSON string,

- hostport : Host name and port of the WebSocket connection
- msgsent : No. of messages sent
- msgreceived : No. of messages received
- encrypted : If the connection secure or not

There is no maintained data available for WebSocket. So all the data related to the connections is maintained at the implementation of the interface(in file nsWebSocketConnectionLog.h) Every time a new connection is made with new host or an old connection is closed, an entry is made or removed from the database if an instance of the interface has been created.

I found websites using WebSocket from www.websocket.org.
Here is one example of JSON string returned from the interface after opening http://rumpetroll.com/ and wandering around a little bit and after that http://www.kaazing.me :

 {  
   "hostport" : ["rumpetroll.motherfrog.com:8180","demo.kaazing.com"],  
   "msgsent" : [76,8],  
   "msgreceived" : [99,223],  
   "encrypted" : [false,true]  
 }  

3) nsISocketLog.idl - This interface returns following information related to current Socket connections in form of a JSON string,

- host : peer host ip
- port : peer host port
- tcp : if the connection is TCP or not

Data related to socket is collected from nsSocketTransportService2.h.
The data returned after opening www.google.com and http://twitter.com/
in above example is:

 {  
   "host" : ["74.125.236.102","199.59.149.243","124.124.201.161","199.59.150.41","124.124.201.161","124.124.201.161","124.124.201.161","199.59.150.7","74.125.236.98","74.125.236.117","173.194.36.30","74.125.236.111","209.85.175.84","74.125.236.118","199.7.51.72","74.125.236.117"],  
   "port" : [20480,20480,20480,20480,20480,20480,20480,20480,47873,47873,47873,47873,47873,20480,20480,20480],  
   "tcp" : [true,true,true,true,true,true,true,true,false,false,false,false,false,true,true,true]  
 }  

This data is not much I am hoping to add some more fields(like total data sent and received) to this interface.
4) nsDNSEntries.idl - This interface returns cached entries in DNS cache with  
    following fields,

- hostname : name of the host
- hostaddr : ip address of the host
- family : ip address family(ipv4 or ipv6)
- expiration : time when the cache entry expires(in minutes)

Well, this interface is near completion(believe me,it is). One little difficulty is stopping me from posting it's output here. May be I post it in the comment after it's done. It's going to contain above fields.

This concludes what I have completed so far.
I have been maintaining a mozilla-central repository at bitbucket.org and committing to my updates to branch 'Networking Dashboard'. You can have a look at it if you want. To test these interfaces, i have made a temporary addon using jetpack without any GUI.

What am I going to do next?
First of all I am going to finish the DNS interface.
Since basic interfaces have been implemented, I can now start working on GUI part of the addon. On the way of doing that, I might need to make differences to the interfaces or add some new data.
I also want to include time stamps, size of data sent/received to above interfaces of different protocols. These two feature are very important to the addon. I will work on them after I am done giving a face to the 'Networking Dashboard' addon.

This was an overview of my GSoC work so far. I have learnt a lot of new things about networking and Mozilla internals throughout this period. Thanks for reading. I hope you download my addon in future :)

Update on GSoC (May 21 - May 13)

I have ported out connection data related to protocols like HTTP,WebSocket and also related to sockets to javascript.

Now I data related to these protocols can be accessed through new interfaces.

Data related to different protocol in particular are -

HTTP -            Host name,  Port no., No. of active connections, No. of idle
                        connections, if connection is using SPDY or not (In future - socket
                        identifiers for particular connections, RTT for connections, TTL etc.)

WebSocket -   Host name, Port no. (In future - No of messages sent so far)

Socket -          Socket identifier and packets sent or received via that socket can be 

                        seen. (Not perfect right now.)

We can search for different HTTP connections for their packets in Socket data via identifiers.

Next, I might add a new interface over all of these interface which combines all of these data and returns a single JSON string. We can also do some calculations using these data.

Update on GSoC (May 7 - May 20)

Though the coding period starts from 21st May, I have started working on new IDL as part of my project.

I started working from 7th May. My first try is to get Http connection data like- hostname, port, No. of active connections, No. of idle connections. In first week I was getting my head around how to create a new idl, xpcom module, etc. Later on I came to know that I don't have to create new module, I can add new idl to Network module. So, now I have created my new idl and implemented it in c++. Only thing lest is to test it and see if I am getting Http connection data in a javascript or not. But it's not that easy. It seams that firefox doesn't allow use of UniversalXPConnect in a normal javascript. So now I have two options, either use jsctypes(which will take me time to understand) or create an add on to access the idl. I am going with the latter. Right now I am facing some difficulties in developing new add on with a nightly build but I will get over it.