EDIT: Hello from the future! This post is part of the old blog. That means it may be deprecated. However, I deemed it valuable enough to keep around. The new blog starts here!

Getting to the stage where you contribute to open source can be arduous. And contributing in itself is a daunting task at first. As someone just starting to contribute to open source, I want to put down now the thoughts I have while in the thick of it and before time blurs the events and the initial challenges or victories.

Getting there is hard

Simply getting the environment required to participate was a first hurdle. It was only four years ago when, after fighting uphill battles to complete my COMP classes' assignments on Windows, I decided to take the plunge and switch my operating system to Ubuntu. Open source is biased towards Unix-like environments (and the popular open source projects on Github are even more so towards Mac OS X). The switch was not easy, but it was rewarding.

After installation, my computer just kept on booting to a black screen and there was no way to switch to my Windows partition... since my keyboard had become unresponsive! Even after fiddling with the BIOS to make my keyboard work on bootup, I still had to figure out how to solve this black screen. And after that I had to figure out how to make my wireless card work. And after that I had to get accustomed to the different interfaces and software!

All of these steps required mucking about with things that even now I barely know about and would rather stay away from. It was quite painful. Oh god, it was sooo easy in 2010 to install Ubuntu. What are u talking about? It was like so much harder before that! <whinny> might say the horse-stradling veterans (we like you guys don't worry). To that I say:

No. It wasn't easy. And it probably still isn't.

The reason more experienced people may think it is easy now is because they understand the infrastructure and tools of today that solve the problems of yesterday. The solutions to the problems they faced during their first time have now had tons of work poured into them. As newcomers we don't know those solutions and even less those problems by definition. To us everything is new and every obstacle is as challenging as it was for the original first time of the veterans.

The wealth of information today has completely changed this some might say. You can find information on any topic easily! Hmmm... nah, I don't think so. At least not the easy part.

We have the opposite situation. Where before there was a scarcity of information -which was at least on topic-, now there is simply too much information on too many topics to be easily grokkable by onboarders. There are so many sites competing for your clicks that you get this cornucopia of patch-work information.

It is very hard to see this for a more experienced person because these solutions are solving past problems which are not even seen as such by neophytes today. I think it is this way because of this pernicious thing known as incremental change. Once you have boarded the technology train, you can see the changes as they are made and how they fit on top of previous choices. In this way you build up an understanding of how things evolve. You build up a little something called experience. But beware: it is the only kind of knowledge that hasn't yet been made available on the other side of a blue link.

All of this stemmed from getting an open source desktop, but the very same arguments hold for getting into the open source world in general. It is especially true for medium to large frameworks. There is often times this disconnect between the maintainers/long-time users' accumulated understanding and newcomers' sudden exposure to the lump sum. I don't like lump sums. They're lumpy.

Participating is easy?...Nah

Choosing a project shouldn't really be too hard. The trick is to use open source projects first. Oh... I see... Well, actually that puts you in an ideal position to discover the problems, limitations or missing features and then you know what to contribute.

You got the revision control software you need? the language environment? the dependencies? the tests' dependencies? the editor? the terminal? some basic understanding of the spoken language of the members of the project? the know-how to navigate these things? some courage? a little lunacy to get yourself out there?

Once you make a list of all the things needed to get into open source, it shouldn't be quite surprising why there aren't that many participants. The technical knowledge required is wide and the time demanded is non-trivial. It gets better with time though. It gets better with experience.

Personally, I was quite lucky to get good feedback on the first pull requests I made. This might also be your case! Others know how hard it can be and so camaraderie is encouraged. Start small and gradually try to give back to some of the bigger projects you use.

Getting into the open source world is a gradual process. That sums up my thoughts on the matter now. I am curious to see if this outlook will hold true as I become a so-called 'veteran'.

EDIT: Hello from the future! This post is part of the old blog. That means it may be deprecated. However, I deemed it valuable enough to keep around. The new blog starts here!

This is it: the final installment of my "On Liberty" Highlights series. You can find here the first, second and third installments. It's been a long and arduous journey - ok maybe not. But it ends here and now with this, the last chapter, "Applications".

Before going ahead, I remind you the rules of the game. This is not a review, a critique or anything so serious. It is a casual cherry-picking of what I found interesting in the chapter.

The good of the many

In this last part, J.S. Mill concludes his essay by talking about a couple of applications of liberty. But before he does so, he summarizes the two important maxims at the core of his essay. I think he is pretty eloquent in his description of them so let me leave the floor to him:

(1) the individual is not accountable to society for his actions, in so far as these concern the interests of no person but himself

(2) for such actions as are prejudicial to the interests of others, the individual is accountable, and may be subjected either to social or to legal punishments, if society is of the opinion that the one or the other is requisite for its protection

The "its protection" part here refers to society's protection. This is actually an important underlying part throughout his essay that I haven't really outlined. The reason why society interferes in the life of others is because it seeks to preserve itself. People are not free to rob each other willy-nilly because otherwise there would be complete chaos and no form of society. It is this idea of protection of the overall society a.k.a the greater good that explains why the government intrudes.

In this mindset, he specifically says the following:

it is one of the undisputed functions of government to take precautions against crime before it has been committed, as well as to detect and punish it afterwards.

He warns about the preventing part as this is the one most open to abuse and thus should be the one we are the most careful about. Hum, hum!

A good application of this (even though he doesn't go into too much detail about it) is upholding commerce and free trade. Even though somebody buying a popularly coveted item prevents others from having it (and thus causes them harm in some sense), the fact that there is this overarching competition is in the end for the best of all. It ensures that the highest return is given to the item producer and in turn spurs the development of alternatives. This is all good as long as it is accomplished in an unadulterated fashion e.g., through cheating, bribery or force. And this is where freedom is set limits by the government to make sure that these exchanges are done correctly (in some cases through preventative measures).

An example: poison (guns)

To bring this back to concrete things, Mill considers the sell and procurement of poison. I think it is a particularly apt example as it mirrors the sell and procurement of guns. Search and replace the word 'poison' by the word 'gun' in the following paragraphs at your leisure!

On the one hand, banning the sell of poison is too much since there are legitimate uses of it (that's right Mr. Rat, run while you still can!). On the other, there is no doubt that poison is dangerous and is the intentional or accidental source of death of other members of society. Labelling poison or poisonous drugs is therefore a necessity to curb the accidental aspect. You can't not want to know the nature (or dangers!) of what you buy - ooh, oh that's right... I guess you can.

To curb the intentional death aspect, a record of the buyer, the quantity and the purpose of the procurement should be kept (even a third party might sometimes be required to attest to the purpose) by the seller. This poses no unbearable burden on the buyer - he/she doesn't need to get an expensive certificate by a professional practitioner - and it can greatly help in dissuading criminal usage of the substance. Plus, it is a safety measure for the seller - it keeps track of his or her inventory.

The level of preventative measures that can be used varies for different things: you need this professional certificate type of deal if you want to get into hedge funds for instance and that's a good thing since you need to know what you are getting into.

Central City

The last thing that stuck out to me was Mill's opposition to an elite all dominating bureaucracy in a government. I think where he really sees government in action is not from the top (bureaucrats) down (citizens) but rather from side (citizen) to side (citizen).

In fact, Mill points at the ridiculous bureaucracy of the Russian Empire of his time and his remarks echo what he previously discussed about China. If everything needs to go through a bureaucracy, then the power has shifted from the people to the bureaucrats. Bureaucracy is necessary to some extent to have people caring about the meta, but these people should be amongst the rest. Not above.

Conclusion

That was fun! It's encouraging to see that a foundational essay is approachable to a layman. Sure, most of the sentences are eight lines long, but the concepts make sense. In the end the work truly centers around common sense. Yay for liberty!

EDIT: Hello from the future! This post is part of the old blog. That means it may be deprecated. However, I deemed it valuable enough to keep around. The new blog starts here!

Yes, it's finally here: the third part of my highlights of John Stuart Mill's On Liberty. If you don't know what I am talking, about peruse the first and second parts. This article will be quite shorter than those two though as fewer ideas stood out to me.

I'll highlight some things found in the 4th chapter this week: "Of the Limits of the Authority of Society over the Individual". Although this is probably the 'most-relevant-to-current-day-society' chapter of the book as it addresses the oh-so-political conflict between the individual and society, there is no surprise or big revelations here: what he talks about is pretty much how the current system works (in practice, we are probably not as even-headed as Mill, but the concepts are there).

Liberty VS Liberty: Fight!

So the situation is the following: liberty is a good thing as Mill argued previously, but obviously sooner or later an individual's liberty will be opposed to another's if our liberties are absolute e.g., I am free to express my opinions loudly, but my neighbours have the right to require I shut up if I do it at 3am with 12 boomboxes. Basically we want a system of living like this:

To individuality should belong the part of life in which it is chiefly the individual that is interested; to society, the part which chiefly interests society.

That's the basic idea and this framework is desirable. There is no other major points beyond this one. This is the framework we should strive for and it is in the making of the laws that the nitty-gritty details get implemented. So according to this setup, you are pretty much free to do anything that won't bring unwanted injuries to others e.g., you are free as a rational adult to bring injuries to yourself.

That's it. Then the text gets quickly bogged down into various corner cases. These are the most intelligible ones:

  • If you hurt yourself, aren't you hurting society by depriving it of your services?

    You haven't signed a contract with society, so it cannot be expected of you to legally owe them your services. Carry on slamming that hammer against your head: you can (weeell, only if you are sane in the first place).

  • This encourages indifference to your fellow members of society! No man is an island!

    First, Simon and Garfunkel might disagree with you on that last one. Second, au contraire this promotes engagement with your fellow citizens / family as it puts the onus on you to actually intervene personally within the bounds of the law. On the other hand, it prevents strangers from affecting what you or your loved ones can do.

  • This undermines social cohesion!

    Again, this prevents legal i.e. state-backed retribution on individuals' moral actions, but it doesn't subvert public opinion. Public opinion is where social cohesion can be found (if this is something that is important to you). Law should not be its enforcer.

This last point is the same one stressed in each of the previous parts: there is the legal world and there is the moral world. Laws are amoral; people are moral (hum...hopefully). And the thing is: people make laws. So it probably never will be so clear cut.

All of this is very confusing because democracy is in many ways a lisp-like language on steroids. The basic elements used to construct the rest can and are constantly changed by votes. It is because of that that so many impediments to the system were instored by the original proponents of democracy: you want to prevent your very basic fondamental elements from changing too easily, but you want them to be technically changeable to account for the future.

Conclusion

That's all I have for this chapter. There was nothing of major note, because this is more or less what we have now. Mill doesn't list limits as much as he gives a rule of thumb (cited above) to determine them.

There is only one chapter left, but I might defer its treatment for a while since that last one left me a bit nonplussed.

EDIT: Hello from the future! This post is part of the old blog. That means it may be deprecated. However, I deemed it valuable enough to keep around. The new blog starts here!

Today I want to rant a lot about documentation. The emphasis will be on software documentation, but it holds for other kinds of documentation too. I chide because I love.

Although the Internet is now this untold well of data and nearly everything comes with 'documentation' attached to it, we've now only reached level 'terrible'. That's just above level 'abyssmal'. Let's be honest; most documentation is somewhat dysfunctional.

Act 1: A terrible experience with documentation

Example 1

Ok. Maybe you don't believe me. Let's do an experiment.

I write Python for my day to day work and at some point I need to see what's the deal with lists. I want to explore all the operations I can do with them, see when to use them or not and so on. I just want concise information on 'lists'.

Hey kids! You can follow along my misadventure from home: fire up your favourite search engine, search for 'python list' and keep the results in a window on the side!

My first result is: 5. Data Structures — Python v.2.7.5 . Here is a dramatic re-enactment of what ensues:

Me jubilant: Great, a link to the official documentation! I will be drinking from the very bosom of Python knowledge today!

clicks, reads first headings

Me: 'More on Lists'? There are a couple of functions here, but where is the "Less on Lists" part then?

looks around, checks the sidebar

Me: Only section 5.1 deals with lists and the rest seems to be dealing with data structures - wait, what is 'More on Conditions'? What does that have to do with data structures?

clicks, reads through

Me quizzical: Apart from the one paragraph among the seven, that has nothing to do with data structures really.

notices the footer

Me: Oh, this is the Python Tutorial. I get it... huh ... actually I don't. There is nothing 'tutorial-y' about this section. Like at all... Whatever.

looks back at the table of content

Me: Previous and next topics are unrelated, let's look at the table of contents for the whole Python Tutorial. It does say 'More on Lists', so lists have to have been covered before.

finds *3.1.4. Lists*_

Me: It mentions 'len()'; it mentions slice... That's all good, but isn't there a concise page just on lists? It is kind of strange that there is no link to the canonical page about lists. Clearly, this isn't complete since there is a 'More on Lists' section. And who knows if that section is complete? Bah! I am looking at the tutorial, I shouldn't be expecting too much! It's for learning. It's not a reference! Let's refine the search.

regains hope, searches for 'python list functions', finds *5. Built-in Types — Python v2.7.5 documentation*_

Me: Ah, yes! This links comes from the library and not the tutorial. It is the real thing. The light to the shadow I encountered before. Come to me knowledge!

clicks, ctrl+f to find the list section

Me muttering: 5.6. Sequence Types — str, unicode, list, tuple, bytearray, buffer, xrange . Hmm, this is not a reference. This is an explanation of various things that apply to these various objects. Where is the section devoted solely to lists?

looks around

Me : Ooh, there is a link named 'list'.

clicks

Me: What. Is. This.

End scene.

Yep, there is no single page in the official documentation where everything about a Python list is, well, listed. Maybe I was asking for something bizarre and elusive. Or maybe not: http://effbot.org/zone/python-list.htm.

This link comes as close as I've seen to what I was looking for: a page solely concerned with Python lists that shows the things you can do with a list and that discusses implementation and performance details. It is only missing a reference of all the functions pertaining to a list. There is also the unfortunate fact that it dates back to 2006. Is it even accurate anymore? And why is this coming from a third-party and not from the official documentation?

Exhale.

Example 1.5

Now this is just one case of a terrible experience. I am sure you've had many more. I was going to refer to Canonical's MAAS (Metal As A Service) and Juju documentation, but I am a fair and balanced critic and it seems like they have improved their game since their latest release. It was slightly incoherent and disjointed before. There is still no explicit link from the MAAS documentation to the Juju one though. And the video is still basically a lie. You will need all those O'Reilly books. Trust me.

Example 2

Let's pick on Sublime Text a little bit instead. The old documentation is marked as deprecated, yet it is essentially the same as the new one. Except that the old one is more complete and accurate and applicable. Compare the snippets page of the old documentation to the one of the new documentation. The new documentation seems to be the result of a bad copy-paste. Thankfully the version these documents refer to is explicit. Oh... Oh, that's right... They're not versioned at all.

Actually it's not even the official documentation. It's the community one. The official one is much sparser on snippets - read: there is no section on snippets. It's honestly somewhat alarming and disappointing for a product you bought. Sure there is the forum, but don't get me started on forums as 'documentation'.

Argh!

Act 2: Break it down

There are a couple of reasons why these experiences with documentation were not optimum (and again I consider those to be well above the rest of the documentation that's out there). In my mind I attribute their failings to missing the mark on these three key components for great documentation:

  1. Targeted
  2. Referenceable
  3. Explanatory

Targeted

Documentation, like code, should be seen through the lens of a user story. Steve wants to know what a snippet is and how to trigger it. There should be a page for that. Alice wants to see how to write her own snippet. There should be a page explaining to her how to do it. Bob is interested in contributing to the core software. There should be documentation for him (whether that is by having the code speak for itself eloquently or having well-documented methods or whatever).

In each of these cases there is a clear goal that is achieved and more importantly there is an audience in mind. From top to bottom: there is the user, then there is the developer and then there is the contributor.

The way this is done varies. There is nothing preventing these different pages to be combined in a single section-divided page or leaving them be in different ones as long as there is a single page that links to them all.

The Python tutorial targets the beginners. Great! The library should target the developers and the in-code documentation should target the contributors. It doesn't work if one has to travel to many disparate pages to satisfy a single user-story. That brings us to the next point.

Referenceable

Conciseness is of tremendous importance in documentation. If the user of your product is at the point where he/she has to be looking through documentation, you don't want to enrage them anymore than they already are by splitting it across many pages. Nor do you want to go to the DOCUMENT-EVERYTHING extreme where there is so much documentation everywhere that the typical person is just drowning in it. You know the saying: "If you want to find a needle in a haysack, you don't help by adding more hay".

Try to bundle up all related documentation. Make it easy to interface with it. Ruby-Doc is a great example of that. It's short, sweet, coherent, efficient and ubiquitously accessible.

Preposterous statement coming in 3...2...1... The linux man pages are kind of bad at this. Each man page is similar-but-not-quite-the-same. Examples? If you are lucky. Options ordering? There is no rule about consistency. Navigability? Hypertext came too late, let's stick to our guns!

Ok, I know. It is arrogant to think that this 40-years old system still in place is flawed. Everything is explained somewhere and with time you understand why it was done that way... or you just become completely enthralled by Stockholm syndrome - your pick. Yet some part of me says that it is equally arrogant to believe that this system is the end-all-be-all of documentation after only a couple of decades. I am pretty sure we can do better.

In the end the content has to be king. That's my next point.

Explanatory

It might seem that I've only been discussing reactive documentation so far, so let me be clear that this is not the case. Quick aside:

Reactive documentation. Defn. Superficial documentation that seeks to address the practical implementation of knowledge while never addressing the theory behind it. see 'How-to documentation'.

The Targeted section might have given this impression, but true explanations should not be precluded from documentation. Explaining why things are such and explaining the theory/design decisions is absolutely crucial. The documentation should be complete is what I am getting at.

You really want to answer the Why of it all. This will actually answer many other questions down the road and it will be beneficial to you too. It keeps you honest. If your documentation is incomplete, tell us. If it should still be applicable in 2013, tell us. Keep this cartoon by the Oatmeal in mind.

With these 3 components in mind you can look at various documentation and see why they sometimes fail or succeed:

  • man pages: Super explanatory, kind of referenceable and kind of targeted
  • Wikipedia: Super explanatory, very referenceable, absolutely untargeted
  • and so on...

These components are probably not exhaustive but it's a start in understanding our documentation failings.

Act 3: Can-do spirit

I don't want to just rave and rant here. What can be done to improve the state of documentation? I am young and inexperienced in these matters so I am not sure.

In fact I am not sure, because I don't think it's just a matter of tools here. Python's official documentation is automatically generated via Sphinx. Doxygen is another great tool that does the job and there are others.

We have some tools. Are we maybe lacking the culture? Finding out something on your own is cool and praised, but formally contributing the discovery back to the original source seems less common. Documentation is almost always the entry point for many open-source projects. Weird. Shouldn't that actually require the documenter to be much more familiar with the code base than any new person could be?

There is a need for editors. People that will go through the generated documentation and create coherent use cases out of them. Making the documentation consistent and adequately self-referencing builds up the trust in it its user will have.

Documentation can often times be seen as an after-thought. It shouldn't. You wrote all this code for a reason. Tell us! We, the public, will be glad to hear about it. We want to know the choices you had to make and the reasons behind them. It might be strange to hear, but we are mostly sympathetic creatures.

Anyway, next time you tell someone to RTFM! Go read them yourself and ask yourself why they had to ask the question in the first place. Maybe they are FM after all ;).

EDIT: Hello from the future! This post is part of the old blog. That means it may be deprecated. However, I deemed it valuable enough to keep around. The new blog starts here!

Yay! Two years have gone-by and the blog is still standing!

In my yearly state of the blog address, I highlight some of the past year achievements and mostly discuss where I want to go forward. It's more of a personal milestone. When I say that, I am imagining a small menhir along a fictitious "Road Programmer" road... and really you should too.

Let's look at some numbers:

  • 21 posts including this one.
  • 5 posts in French
  • 4 longer, essay-like posts
  • No more git commits (heresy!)

During and in the months following the last state of the blog, I outlined the goals I was aiming at this past year. Here is a quick reminder of them:

  • Post every 2 weeks (Changed from post every week. Also includes foregoing the minor updates post)
  • Post essay-like posts more often
  • Work/complete a side project

The only legacy goal from the first year was to post in French 1/4 of the time. "Achievement unlocked!", may I say.

Assessment

Posting every two weeks got short shrifted. Because I decided to stop posting minor silly updates, it became much harder to pump out content every week. While this did allow me to focus on getting more significant posts through the door (notice that about a fifth of the posts were more essay like and I am more happy with those than the ones of the previous year), it also simply meant less posts. However, I pursued weekly work on the site even if there were no posts to highlight it: CSS fixes, Google Analytics, Pelican transition and so on are examples of that. I address below how I will approach posting in the future having had that experience.

Post essay-like posts more often. I mentioned this above in passing, but it bears repeating: I upped the ratio of meaty to light posts this past year from a 1:16 ratio to a 1:5 ratio. I am pretty happy with that endeavour and I will continue on this path.

Work/complete a side project. I've definitely worked on many side projects - whether 'side' here means for school or work, or whether it means for personal satisfaction. I've completed some, but I would like to finish off some more. I would also like to focus my efforts. I often times suffer from the "Oh. Squirrel!" syndrome. Whenever there is something that doesn't work like I would like it to, I add it to my mental list of things to do. And over time this list just bogs me down. I never act on it and the more I learn the more I realize that those are hard things to fix both from a technological point of view and from a usefulness point of view - one's expected default is another's problematic default and so on. The reality might be that my fix is not only not useful but just additive to the ambient white noise and thus unhelpful to others.

On a more positive side, I have started making very small and sporadic open-source contributions. This is very encouraging as I have so far not have bad encounters and the process has made me learn a ton. I will try to do a post on that later.

The Goals

  • Converge all my programming work to at most 3 languages this year: Python, Nimrod (EDIT: Nim) and something else (we'll see)
  • Write even more (quantitatively) essay-like posts
  • Write post not just as filler but as thoughtful endeavours
  • Select 1 or 2 side projects and stick to them all year/until completion

So the coming year will be more about focus. I've decided to forego the posting quota and direct my energies at doing higher quality work. I also want to do some real progress in accomplishing programming stuff.

It has been an alright year all in all. Let's carry forward!