Tuesday, August 5, 2014

Languages..

I love coding.. The cool thing about coding is that it gives the developer the power to create stuff which can affect lives or solve a problem for a hopefully growing set of people and probably bring up smiles..
I've learnt and played with various languages.. I smile at myself when I think of the attitude that I had  towards languages.. Its changed radically..
  • I learnt C during my college days and loved it, enjoyed it thoroughly..
  • During college times, I was like. C is the only language I'll learn and I'll code anything I want to and everything, in C itself.. So I used to explore how to call run other code written in other languages like Calling Matlab code from C and vice versa.. Those were the pointer* days.. The idea that I'll do everything in C itself.. was a severely flawed notion that I had in mind which I've realized in the times to come..
  • Then came my job at Solidcore.. I was introduced to C++ another interesting language.. But in some ways a funny language coz of the excessive amount of control which C++ gives in terms of speed and OOPsies, the syntax seems to have become in some way complex coz of the backward compatibility with C..
    • An example of the control which C++ gives is like by default it gives you Compile time binding of methods but by using virtual you can impose late binding which has its overhead.. Now its upto you to decide whether you want to compromise on the extra instructions that are going to be there for polymorphism or you want the speed.. whereas in Java one would find dynamic binding itself.
  • Slowly I started getting interested in web and the associated technologies aka the J2EE stack for the backend. So I was introduced to writing enterprise solutions in Java. Java today is a language which gives you:
    • robustness - Its a compiled language, type safety, Brilliant concurrency support, Some of the most well thought out system libraries i.e. Collections etc.
    • scale - Coz of the inherent OOPsy nature.. 
    • speed of development coz of the brilliant editor support. Oh yes, Eclipse..  
Again as a result of my curiousness to check out the interfaces with the native world etc. I was introduced and did quite a bit of JNI coding.
  • You cant just develop a usable web product by only doing the backend.. So next comes Frontend and with it comes JavaScript.. One word which comes to my mind when it comes to Javascript is versatility.. Write any code and it would work well just in case its few lines.. But then, in today's world with the extra thick frontend layer and the many lines of Javascript that we got to write, if you're irresponsible with Javascript you end up with spaghetti code, which is a maintenance nightmare .. One has to be super disciplined. The good thing about Javascript is that if you know the patterns well, and the use the flexibility + versatility that the language provides in the right structured way, things work great and your code scales and is maintainable.. Add linting + powerful frameworks(AngularJs, JQuery) and your frontend is now taken care of.. 
  • You need to know a good scripting language.. Back in the days when there was no node.js I was introduced to Python.. A language which is elegant in itself.. Wrote quite a bit of code in Python and it was a thoroughly enjoyable experience.. Sometimes Python just reads like pseudo code.. Any piece of logic, file manipulations, etc and Python does the trick.. You understand the significance and power of the dict = {} and list = [] literals.. And how one could solve many problems.. I've mostly written one file python scripts which solve a simple problem to a fairly complex problem, like a tool or something.. But a whole product built on this would again mean learning frameworks + tools etc. Somehow Java and the J2EE stack fulfilled that need for me. Another point to mention is regarding the packaging of your scripts/tools.. Packaging the script as a binary using libraries like py2exe suddenly makes your script so very shippable, installable and usable for windows users.. 
All along, I've learnt that one needs to know their tools really well. The more you play with them, with their features the better you get at them.. After that all you need is a great product idea, or actually a few great ideas, (Coz, many of your ideas sound great, specially to you,) Out of the many great ideas that you execute successfully upon, finally, something will end up being great for hopefully many users.. And I, as a programmer could speak, interact and communicate with my users, via these very languages.. :) :)

Sunday, December 1, 2013

I love running

Yes I've fallen in love.. Unconditional love.. 

I love running.. Running is like meditation to me.. It's time devoted completely for me, where you get to talk to your inner self, when there are those self realizations.. Running is  like dope.. Makes me go high, super high..

Running helps me realize that deep down there's a sportsman in me, someone who's not going to give up, the 'never, ever give up' attitude gets revived again.. The thoughts that you're going to do all you can to set things right.. The fighting spirits echoes loud and strong.. You realize how lucky you are to have whatever you have. Blessed I am to have people like Gautam, Ashish and many other wonderful champs, this space-time continuum has introduced me to, the ones who would stand by me whatever happens and patiently bear all the bull crap 'yeh-woh, aise-waise' talk that I do :P.. 

When I run, poetry starts making sense:

पिघले  नीलम सा बहता  हुआ येह समा
नीली, नीली सी खामोशियाँ
ना कहीं है ज़मीन, ना कहीं आसमान
सरसराती हुई टहनियाँ , पत्तियाँ,
कह रहीं हैं , की बस एक तुम हो यहां
सिर्फ़ मैं हूं,
मेरी साँसे हैं, मैं और मेरी धड़कनें
ऐसी गहराईयाँ , ऐसी तनहाईयाँ,
और मैं, सिर्फ मैं ,
अपने होने पे मुझे यकीन हो गया.

I feel alive when I run and that is what matters, that is what makes me fall in love over and over again.. :)

Wednesday, August 28, 2013

Realizing nature's paradox: Uttarakhand

Last year 2012, I had roamed in and around the Valley of Flowers, Kuari Pass, Badrinath, Hemkund Sahib.. Those memories and the immense beauty around, they are still fresh in my mind..

The current devastation around these same places makes me wonder..
How the very same nature can contradictorily strike back with all its fury... Hurts to think about the condition of the people who are caught up and are out there suffering..

Tomorrow its going to be another trek into the mountains, into the Himalayas, with some amazing friends..

Coz as its said..

"The core of man's spirit comes from new experiences. "

And ultimately in Lord Byron's words..
"There is a pleasure in the pathless woods;
There is a rapture on the lonely shore; There is society, where none intrudes,
By the deep sea, and music in its roar;
I love not man the less, but Nature more..."

With the hope that nature and the mighty Himalayas.. would shower on us with all its grandeur and beauty..

Into the wild

Of the very few times that I gather the patience and effort to complete a novel.
Mostly I end up reading non fiction. And there's a solid reason behind that..

Even though I respect all the amazement, that the imagination in fiction can produce. But, nothing can beat the purity that real life experiences has to offer. There's some assurance that the stuff actually happened or is concrete..
Also, There's awesomeness in what real life can show you or the variety of experiences and things different people and cultures go through.

Into the Wild is a book I wanted to read since long. I had watched the movie and I liked it. But a sub 2 hr watch wont answer all the questions. There are these lots of Whys and Hows? The Perspectives, that were missing.

The book does answer many of them.
It does explains to a huge extent the state or the thought process of a person who wants to shun society and wants to be alone for months together or maybe for life itself. My love for nature and the mountains gets reinforced when I try and relate with some of the thoughts presented. One gets to learn what not to do when you're in the wild. The final learning for me is that there's this yin-yang balance thing about obsessions which should never be broken. The complementary Yin's and Yang's should ultimately lead to happiness.

Tuesday, May 29, 2012

Joys of Spontaneity: A two wheel ride to Kaigal falls

Its year 2006, and its one lazy afternoon.  I had just finished with 2nd year engineering and it was summer vacations time. And the place is Kolar Gold Fields (KGF), some 100 kms near Bangalore.
Oh my, it’s been eight years now. But this is one trip which is all so very clear in my mind.

Srinidhi, my school pal had come home. Initially we were chit chatting. We had lunch. Slowly boredom creeps in. And that calls for some action. During school days we had heard of a waterfall nearby. Kaigal falls. We got to do this. Let’s go to this place, is the call. Now comes the planning phase. First of all how far is it? Ok that we both aren’t sure. So we call up some pals. Some say its 20kms some 15kms.

Its 3pm now. And the plan is clear. Once we start we should cover the 20+kms by 5pm. We'll hang out there till 6 or so. And then return back home to have some good food.

I start of on my bicycle which is the same one which I used to ride to school. It’s a Hero Devil which has gotten quite rickety now. It has no bell no lights. But then there’s something about your first love. Right? And sure I loved this two wheeler. 

My almost brand new Hero Devil :)

No bell, no lights not a big deal because the part of the ride where there would be no streetlights should be covered in daylight itself i.e. by 7 itself. And the brakes are pretty good even now. Ditto said my friend. So we start off for the ride. All excited. As always. :)

So en route we already knew that we are going to pass by a small town called VKota. And this water fall should be somewhere nearby this town. We keep on riding some 15 odd kms to reach VKota.. We inquire from the locals in VKota and they are like Kaigal falls should be another 20 kms from VKota.. When the first person tells us this we just think that this person doesn't really know about Kaigal falls.. But it so happens that after every 10 minutes that we ride we keep on hearing the same thing. i.e. Kaigal falls is another 20 kms to go. Its already 4.30+ so we got to decide what needs to be done and recheck our plans, That is, we either return back home or keep on riding and reach Kaigal itself.

We are like now that we've come this far we should just go to Kaigal falls. It’s a waterfall. So there should be a hotel or some facility to camp out there, where we could stay for the night and then return back the next morning. After all, it’s a famous tourist spot was the reasoning. ;)  So I call home from a PCO booth informing that I wouldn’t be returning home on that day. Next issue to be solved is food. We don't have anything to eat and drink. So we buy two bottles of water, two glucose packets and biscuit packets as we set off riding. After passing by VKota the views around gets really picturesque. I dint know the name of the place then but googling for it now it’s probably Koundinya wildlife sanctuary  that we were passing by. I tell my friend that probably we could come to this place itself some other time and camp here. Coz this looks like a nice beautiful place to trek. We are sipping off the water and pedaling the two wheeler away to glory. It’s about 6.15 now. And it’s quite a forest-y kind of an environment. Very few vehicles around. Lot of birds chirping. At times we do hear some voices which do sound like forest-y animals.

Some road signs and inquiries lead us to a mud road to Kaigal falls. As we approach near the falls, nature has DJ'ed in lot of music into the background. There are birds chirping as they return back to their nests. Different kinds of monkeys also seem to be in the mood to riaz(practice music) off, their vocal skills. The sound of the water falls is getting louder and adding to our excitement. One issue is there doesn't seems to be a soul around. Instead we do spot rabbits, hares hopping and running around. Cool it is. Riding along the mud road we reach the spot beyond which we can’t ride a cycle. Hmm, so we walk a bit and finally spot the falls. We are so very happy. So after raising a glucose + water toast to the beautiful place we decide that as its getting dark we should first find the hypothesized hotel/camp site to spend the night through, before playing around in the waterfall.

After some exploration of the place around we are yet to find people. Also our hypothesis about hotels and accommodation around seems to be completely wrong. But we were relieved to spot a bungalow kind of a place nearby. And lucky for us there's a bulb outside the house that’s glowing. We reach there and we start knocking the door but there isn’t any response. After about 3-4 minutes of trying to find a soul there and constant knocking with awesome natures background sounds. Nobody comes out. No human voices heard and to our horror suddenly the bulb switches off. Damn! We both are like running as fast as we can to our cycles. And we don't stop till we reach the end of the mud road. Either there was a scared soul inside that house or there’s immense potential of lot of TV serial like stories around what happened.

Its almost 7.30 now. And we again start planning about what needs to be done now. We decide to ride till VKota town and stay there for the night. And with no light source in both our cycles we knew it’s going to be tough. Also the road had an interesting condition, the sides of the road were very steep. So just in case we slipped from the tar road to the muddy part on the sides, while getting back to the tar roads there were lot of chances that one would fall, depending on the kind of tires, which was again screwed. Hence, both of us fell. I was the slow cycling champion at school but when I fell I realized that after a level we are slaves of the kind of terrain we ride in. There was no light at all. We had to rely on the few vehicles with their headlights on, that passed by to get an idea of the road that we need to ride by. Slowly and steadily we battled out and reached VKota at some 10 or so.

Weird town/village VKota is. They seem to religiously follow the first part of the "Early to bed, early to rise  ..." quote. The few hotels which were there, were already closed. We did wake up some of the hotel owners. And enquired about the rates and availability. We considered the option of taking a bus to home, as there dint seem to be enough value staying at a hotel there, the hoteliers there dint seem to be interested in hosting us either. There were quite a few policemen in this place coz of the area's geography. This place is in Andhra Pradesh and is border to both Karnataka and Tamil Nadu. The policemen were quite suspicious but they suggested to take a bus back to KGF. So we waited at the bus stand to find a bus back to KGF. Out of the few buses which were going to KGF most of them dint allow us to load the cycle on bus and take us along. Finally at around 12 at night and some amount of convincing we got lucky. We put the cycles on the bus. And the bus went on at a wonderfully slow pace. We dint mind at all. We were too tired to talk, both of us took a nap and reached KGF bus stand at some 1.30am or so. It was another 7-8 kms from the bus stand to my home. But this was a well-lit route. We reached home finally at some 2.30 or so. I dint want to knock the door to wake everyone up and get all the scolding’s. So, I went to my brother's room's window and I called for him to open the door for me. He, like a good, sleepy, zombie boy just opened the door dint ask any questions and went back to sleep. :)

I gathered all the stuff that's there to eat at home. We finish hogging. And finally it’s time to sleep. Amazing day of adventure it’s been. It’s been scary at times. Had the share of fun moments. Lot of good decisions, lot of crazy decisions. This was the first of the marathoner kind of an event for me. I had cycled more than 70 something kms in a day for the first time. I almost felt like superman! Finally it’s time to tell goodbye to the day gone by. I lie down in the bed. This is the wonderful quote which sums up my thoughts about life and such adventurous events that happens in life.
“Life should not be a journey to the grave with the intention of arriving safely in a pretty and well preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming "Wow! What a Ride!”
― Hunter S. Thompson(Probably)

Friday, July 15, 2011

I scrape as a timepass, and its fun

'Scraping' the name itself sounds negative even though in my view Scraping is a real boon to developers when there's no API(REST, XML based, etc) already published.. REST APIs are very widely adopted. Lets discuss something about REST first:
REST is basically like, there are some resources(objects) on a server. Now these resources can be accessed via the same old HTTP GET, POST, DELETE... (maybe after authentication which these days is mostly via OAUTH). In response to the HTTP request the server sends back the JSON string. The gud thing is JSON is quite readable and can be parsed very easily. So the resources/services on a server can be communicated with the clients via the HTTP requests and the JSON responses.
Lets compare that with scraping..
  • When we want to scrape there's again a HTTP request to a server.
  • Server sends some HTML, CSS and JS.
Now there are issues with the HTML and the JS content sent.
HTML issues:
  • Browser seem to handle any hopeless HTML  sent.
  • So the content becomes tough to parse coz people i.e. the HTML content writers seem to be sloppy.
If proper HTML standard is followed by every content writer then its very easy to parse it and use it.. but when the HTML code is gone case we rely on libraries which heuristically parse and give you good searchable objects to enjoy with programmatically. The logic that the libraries would work is fairly simple ..

If browser's can parse hopeless HTML to a degree that the browser still renders bad HTML good on it, implies there is parsing code available with the browser. Mozilla Firefox source code is open. There's lots to learn there..
But to get a quick scrape done we really don't need to look into the browser's HTML parsing code. Instead we've got people who've made great libraries probably looking at the browser's HTML parsing code.
For my scraping needs I've enjoyed with BeautifulSoup when I code in Python and Jsoup when I code in Java. Playing with these libraries is sheer fun.. To get a feel for how good beautifulsoup is check this code that I wrote long time back. This just gets all Indian railway train information data like schedule of different trains and train stoppages from http://www.indianrail.gov.in/ and makes SQL queries to persist the same data.

Ok so now that we've discussed that HTML parsing is more of an issue solved, lets talk about the JavaScript part of the HTTP response..
  • Scraping Static pages is easy as explained above. 
  • Today pages are dynamic. And ajax calls really make the user experience better.
So now if we are scraping a dynamic page we need to understand and render the Javascript code too.. Whoa.. But just think of it in terms of the browser..
"The browser understands JS and Ajax and renders the content received from AJAX calls."
But this case is very different and is far more different in comparison to the HTML issue.. The JS code interacts with the HTML and CSS so now we need to emulate the whole browser itself. That is when COM(Component Object Model) objects enter into the picture. We basically have to emulate the full browser and COM helps in doing that. Check out how to programmatically use Internet Explorer COM objects with some of the python code:
from win32com.client import Dispatch
ie = Dispatch("InternetExplorer.Application") #Do stuff with the ie object
 Ok so scraping the dynamic pages is tougher in comparison to static pages.. But it can be done. 
Also many a times one can get the right data by finding out the URL to which to ajax call goes and then use the response received. 
There's still lot of functionality which can be achieved from many of the static pages still available.. 
For example:
Visualizing realtime stock data is crucial for traders. Now in many of the Stock Tracking websites available the stock data being viewed is some 2 minutes behind the actual time. Also there are websites where a HTTP request to a server for a Stock returns the realtime price of it.. But the problem there is that pages have to be refreshed again and again to check the current price.. Also in one web page in the browser you can check only one Stock at a time.. 
So me and a friend of mine(he is a big market lover :) ) tried to solve the above two issues and these were the minimum functionality that had to be solved:
  1. Choose the companies from NSE that the user wants to track. 
  2. Aggregate all the stock prices of the companies in one page.
  3. Auto refresh the prices via AJAX every 10 seconds or so.
So we made a Simple and basic Realtime stock monitoring application which achieves the above, this uses GWT and the Google App Engine to host which are some interesting things to talk about.. Probably for some other post. Even though the User interface isn't good but still it serves the purpose.

Ok so scraping Static pages can be visualized as an API itself, albeit the fact that the content will change with time.. And the chances that the scraping code breaks when the content changes is always there.. One small tip to avoid breakages to some extent is to use CSS classnames if they exist while scraping, that way hopefully if the page author wants to restyle he/she would change the CSS properties of that class instead of the class name..

Finally Scraping is fun for developers like me coz it helps solve many interesting problems.. But think in terms of  how the search engine giants Google etc. work . They rely big time on scraping and crawling.. Their bots have to literally crawl the whole web and scrape data based on their algos to help their servers index and search.. So scraping itself is a huge revenue generator too but for now I will enjoy the fun part of scraping .. Hopefully some time in the future I scrape to create revenue.. :)


P.S. I scrape just for timepass.. Please dont sue me.. ;)

Saturday, May 28, 2011

Serendipity..

This post is about an interesting event that happened last weekend..
Its 21/5, ~10pm.. I am returning back from Airport to Gurgaon.. The following are the sequence of events that take place..
  • Me in a shared cab a pretty gal sits next to me..
  • The cab goes from airport to Gurgaon
  • Lots of silence initially...Finally when gurgaon seems to be nearby she asks the driver about some hotel.. Driver doesn't know..
  • I step into the conversation.. She tells me about the place.. Even I dont know.. but I tell her to ask Justdial and give her the no.
  • Fortunately or unfortunately justdial doesn't give out any concrete information..
  • So I call up a friend of mine to know where exactly this hotel is.. And he knows..
  • I tell her about it.. 
  • She agrees to take an auto with me to the metro where.. I had kept the bike, so i tell her about it.. I tell her that I'll drop her to that hotel on the bike .. she agrees.. 
  • So after the auto ride and some talk we head to the bike..
  • Even on the bike I get to talk to her and all.. and even she's like talking rigorously and all, like lot of the gals do..
  • She tells she is an airhostess and is new in gurgaon..
  • She's searching for a PG..
  • The hotel arrives..
  • She says bye..
Eventually I dont get to know her name.. Also I could've given her my cell no. coz I could've search for PGs on the internet and help airhostesses you know.. ;)

But I'm just happy I helped a gal who was lucky enough to find a decent guy like me.. at the night.. or else things can get pretty dirty and bad here in gurgaon/NCR when a gal who's alone meets gone nut case people at night..

Such interesting things should keep on happening.. But I shouldn't be screwing up as I did this time..