Wednesday, 19 June 2013

Learn to encode, please!

As a software developer, you should be aware of character encodings and avoiding several problems with encoded strings.

Thanks to many software developers' issues in understanding that (and Windows) (and paths with spaces), I have four home folders in my system.

Thank you for your understanding. Or should I say, please start understanding?

Friday, 17 May 2013

Undercutting - a personal rambling

My underpaid self has been thinking about his own life and actions. It turns out I am doing the wrong thing by sitting in the same place.

I have spoken to a friend about my salary and what I do at work, and she scolded me for helping undercut other people's well deserved salaries. She is quite right. She told me of how she and her group had lost a web and graphic design project to some company who promised to do the same for much less. They obviously ended up doing a terrible job, and screwed up someone else's opportunity in the process.

My current employer does do the same thing, but this post is not about companies undercutting each other. It is about workers undercutting other workers in the same sector. By allowing myself to be underpaid for my work, I am not just being a fool. I am undercutting other IT professionals who live around my area and have skill sets similar to mine.

When I started working at my current job, I was told that my pay would increase as soon as I had proven myself a valuable addition to my company.

I should have known it wasn't true as soon as I had learned that a colleague and old friend, who is a skilled programmer and most defenitely a valuable addition to the company, has worked for nearly three years in the company and earns little more (less than 1%) than I. It should have been blatantly obvious when I knew that another colleague, an IT tech and nerdy nice guy on whose shoulders the company runs a lot of its profit and stability (he runs the actual maintenance operations for the company's maintenance clients), is in a worse situation, since he actually deserves more than both of us, but earns the same amount we do.

And as time passed and I created websites, applications and whatnot, it was obvious that that "valuable addition to the company" line was just crap.

I feel like a sucker. Now my country is in a crisis. Prices going up, a higher tax on my salary. My boss is saying that if we ever get a raise we will only waste the extra money in luxuries. I guess that's how he sleeps at night. This shit revolts me deeply.

But I never stopped to think that I was actually harming someone else.

Sometimes people are not too picky about their jobs. People at the start of their carreer or people who have just lost a job are often happy to take whatever they get. Some companies (like my own) are opportunists and will use this instead of hiring qualified personel for a proper value. Some companies use excuses like "gathering experience", "prove oneself as valuable to the company", or "finantial issues" to postpone expectations of better pay for another day. Empty promises that I am sure have made many boost their dedication and productivity for nothing. Actually they ended up undercutting a lot of people and yet deserved better pay themselves. It happened to me. I have seen it happen to a dear colleague who was hired after me.

This snowball of bullshit has to stop, and not just for me and my company. If you are in a similar situation, you should at least consider trying to end it if you can afford losing your job for a few months. If everyone accepts crappy jobs and bullshit, someday we all will only have crappy jobs and bullshit, and nobody will be able to do anything about it.

Tuesday, 14 May 2013

Python's new Enum class

I used to wish to have an enumeration in Python, like you have in other languages.

On May 10th, GvR accepted PEP 435, which proposes the addition of an Enum type to the standard library.

Before this, us python coders would need to use plain classes for enums, which doesn't work so well. We can't get enum names by value, for example.

It follows that it's not possible to create a list of 2-tuples for use with Django models. We'd need to have a class mapping our choice names to their values, but you can't create the good old "choices" structure, so that's pretty useless. However, since our new enums are iterable we can:

    class ChoiceClass(IntEnum):
        foo = 1
        bar = 2
        baz = 3

    CHOICES = [(e.value, for e in ChoiceClass]

Creating your enumerations

By inheriting from Enum, you can create your own enumerations. If all your enumeration values are supposed to be of the same type, you should inherit Enum as well as that type.

    class Numbers(int, Enum):  # you can also use IntEnum
        one = 1
        two = 2
        three = 3

    class Text(str, Enum):
        one = 'one'
        two = 'two'
        three = 'three'

Using your enumerations

By iterating over your enum class, you get the enumeration members, which can be queried for their name or value.

    print(' '.join(['{}: {}'.format(, number.value) for number in Numbers]))

Internally, there is some metaclass magic going on, to allow us to succintly declare these enums. The class comes with a few facilities, like __iter__ as I showed above, __call__ which gets enumeration items by value, and __getitem__ which gets them by name.

You should understand that enumeration items are actually instances of your enumeration class. Which allows us to say isinstance(, Text).

You can get your enumeration items in several ways.

  • As an attribute:, the Java way
  • By name: Numbers['one'], a tad prettier than using getattr
  • By value: Numbers(1), as a constructor

In a way, the third syntax reminds me of how we convert values using their constructors in python, for instance, using str or int to convert existing objects to string or integer values.

You can start using these enums right away. There was talk of porting this back to python 2 on the python-dev mailing list, but right now the code is for python 3 only. I'm going to convert some Django code to use these enums later, because I have three declarations for each Choice I need (SOMETHING_CHOICES, SOMETHING_DICT, SOMETHING_REVERSE), which is just backwards.

You can grab a copy of this module at bitbucket. Beware, because this was only put up for PEP evaluation. You'll want to wear a helmet and keep in mind that this is not stable or supposed to be used in production code. Although Guido has accepted it at the time of writing, it may still be subject to change, since the PEP is not in the "final" status.

That said, grab the "ref435" module from that package and try it out.

    >>> from ref435 import Enum
    >>> class Enumeration(int, Enum):
    ...     potatoes = 1
    ...     apples = oranges = 2
    >>> Enumeration.apples is Enumeration.oranges

You can look for reference in the PEP.

See you next time!

Tuesday, 7 May 2013


Yesterday I have had the joy of writing some php code. It was marvelous. minutes later a syntax error popped from a regular function taking a string. I called my colleague, who had experience with php. We were both stumped. It used eval. I guess we are both dumb.

Php is so simplistic and easy to understand, it's painful. If I pass an undeclared constant off to a function, php goes all the way to interpret it as its own name for me. I guess that would make some code more readable. Why mix foo with "bar"? This syntax allows us to just go all the way and use constant name syntax everywhere. Then we just don't declare the constants supposed to be those strings. Such advanced magic.

They say that the best technology is indistinguishable from magic. I say the best technology goes ahead and makes my code run more easily, even if it has bugs which will pop up later. It's so smart right? Who cares about tomorrow? Everybody knows code is written just once and doesn't need to be maintained or anything like that.

Code for today. Write no abstractions. Php allows the ultimate freedom. If you want to shoot yourself in the foot, just do it. It's not like there is anything stopping you at all. It's your job, not mine. You know what you are doing. We all know that php is a language for experts, everyone will know better than to carelessly use unsanitized user input to form SQL queries (and HTML too!) It all comes from $_GET clean and safe with your magic slashes right?


It's not a language problem as much as it is a cultural problem. Although the former appears to have caused the latter.

Sunday, 31 March 2013


When testing my views in a Django application, I always use django-webtest. This is a Django binding for a library from paste, which simply talks to a web application using wsgi, and acts like a browser in a very basic manner.

For example: submitting forms works as expected and sends the data contained in the inputs inside such form, in the method defined in <form method=""> and to the url defined in <form action="">, click()ing links takes you to the url in their href attribute, etc.

These are tests which will actually tell you if your templates are broken. So you are not simply testing your views. You are testing the template html as well. If you forget to add that {÷ csrf_token ÷} to your form, the test will fail. If one of the designers accidentally removes your "edit profile" link, you will know that.

An obviously missing feature is JavaScript support. So, if your app doesn't work without it, you won't be able to use this to the fullest.

Your view testing code with WebTest will look like this:

    def test_delete_object(self):
        object = construct_model_instance_somehow()
        owner = object.owner

        # go to the object's page, logged in as the owner
        details_page =, user=owner)

        # click delete link
        delete_page ='delete')

        form = delete_page.form


            some_data='you know').exists())

Every .click() call is an assertion that the page loads with no errors, and returns the response. You can actually chain click calls, even though I don't find it so useful myself. If you want a BeautifulSoup to check that your HTML response contains the correct links, just install BeautifulSoup (3.x for now) in your environment and access response.html to get a soup. Not to mention .showbrowser(), which fires up your browser to show you the HTML directly. A huge time saver for debugging.

In conclusion, very useful stuff. You should use it.

Monday, 25 March 2013

Pip requirements.txt tip

Good guy requirements.txt

Everybody loves requirements.txt files, right? You can list your dependencies, and then together with pip and virtualenv reproduce your application's environment complete with all dependencies in several machines.

This file is also known as pipscript.txt. in case you don't know, requirements.txt is a file containing a list of packages in this format:

    -e git://some.git/repo/containing/package

It's very simple to use. You just have to use the command pip install -r <requirements file>. For this reason, it makes it very easy to automate a deployment step.

Splitting your requirements

In case your application needs to split requirement files for some reason, you can do that! Just add this line in a pip file:

    -r some/requirements.txt

To include another requirements file.

The requirements listed in the included file will get installed too.

I found out about this trick in this heroku doc page . It makes a lot of sense, seeing that requirements.txt represents a pip install command for each line. We get -e, why not -r?

This is very useful for Django, where you need a different package set in production and development.

Monday, 18 March 2013

Launching Q&A Website

In the next few weeks, I will be deploying and starting a new website. It's a Q&A website with focus on bureaucracy. It is localized in Portugal, so I won't be writing much about it on this blog.

I'll be using question2answer, which is a PHP-based open-source QA website. I will customize it and open-source that customization on github as well as a few fabric scripts I will use to test and deploy the website.

There is no kind of business plan yet. I don't really intend to make more money from it more than the amount needed to pay for the hosting, but this could change if I saw this really taking off.

As for marketing and getting people on this, I have started a Facebook page last week, with not too many people on it so far. I'm guessing I will need a lot of people to be there before I get to launch day.

Nobody wants to use an empty website. I have looked into how stackexchange launches a new website, and it's very interesting. They have the area 52 site, which is a staging zone where ideas are tested. People can commit to proposed sites and example questions are added. If a site is deemed good enough to launch, it goes into beta, and then becomes a full-fledged website.

I have to learn a bit more about stackexchange's process. I'm not too clear on the specifics.

Anyway, a simple "this is coming soon" + FAQ single web page is up in its future domain,

I never started any kind of website before, so this will be a great learning opportunity. I am looking forward to it!