A cube-farm of painters

Programming is a truly creative art. Sure there's rare times when you could hire a monkey to pound on a keyboard for the same effect, but software is written on computers by engineers who can make computers do simple tasks for them. This means that any repetitive, dull, and automatable task is usually relegated to the computer. All that's left is the creative tasks.

A lot of software departments don't seem to understand this. They think of programmers more like accountants or telemarketers where the work is fairly dull repetitive and supervision is necessary to make sure everyone is working all the time.

I'm not saying that a room full of unsupervised software engineers is likely to spontaneously produce great software, but a programmer won't thrive in a space designed to keep an eye on how much programming they do. They need some space so they can foster their creativity.

As a metaphor, lets consider how one should design an company if that companies product was going to be a huge piece of art. I'm talking so big that you'll necessarily need to hire multiple painters and will have to make sure that they coordinate their work. The nature of the piece is such that it all must flow together and can't be easily broken into different parts.

How would this work ideally if the artists were in charge? They'd meet at the site of the painting, and have a brief talk about what ideas they have for the piece. The painters with more experience would likely have more to say, but everyone would contribute. Then they'd all start painting sketches for parts of it. Later, they'd all step back and have a look at how it was coming together. Some parts would look good, others would look bad. The bad parts would be discussed, and new ideas would come forward. The best artists would spend their time on the parts that were proving to be the most difficult, while developing artists could hone their skills working on simpler parts of the design. Parts of the piece that were critical to the design would be discussed in great detail, and would be implemented by the most skilled artists, while less important parts would be left up to individual artists to implement. If certain parts required techniques that a certain artist was an expert in, they would be the most likely to work on that area. The project would move along until the artists agreed that they didn't need to add any more and it would be done.

Now let's consider how the typical corporation would approach the problem. First, the oldest artist would be promoted to "art manager" and the second and third oldest would be made into "art supervisors". Everyone would get a cube, except the manager who gets a small office in the corner. The three elder artists would meet in a conference room and decide what they thought the overall design would be. They'd then decide on a line to divide the project in two, and each supervisor would be assigned a part. They would also come up with a timeline for each stage of the project, and would pick a date to complete the project on. This date would be based on the opinions of the three elders, plus a buffer of time for unforseen problems.

The management team would then host a big meeting where they explained their vision to the rest of the team, and explained the schedule. Each artist would be assigned by his supervisor a piece of the project, and they would sketch up a design for it. The sketches would be brought to meetings where they could be compared to other sketches to see if they looked good together, and to make sure that they fit together well. The supervisors would collate the sketches and present their halves to the manager. In a good company, the three elders would spend a great deal of time discussing how the interface between the two halves would be completed.

Finally, when the schedule indicated the time, the team would make their way to the project site. Each artist would try to implement their idea onto the project. If another artist was in their way, they'd have to wait. If their sketch didn't exactly fit onto the project, they'll fake some way of making it fit, but since they don't know much about the overall project they can't anticipate how that will affect others. The borders between each artist piece would be implemented by the artists involved, and any disagreement about the border would be decided by management.

Parts of the project which were proving to be difficult would end up with lots of management overview, although the implementation would usually still be done by the artist responsible for the part. Rarely, the supervisor would switch an area between two artists if necessary to meet the schedule, but a supervisor would never let his best artist work on the other half of the project, even on a difficult part that could use his expertise. The manager would spend as much time as possible looking at the overall state of the project, but since the most important and difficult parts are typically the last to get implemented, the overall view remains unclear until the end. The manager will also be involved in all discussions about how the two halves of the painting should be combined. But the manager will never actually paint on the project regardless of his skill or expertise as his time is too important for such things.

As the end of the schedule nears, the most difficult parts will be the only parts left undone. Since there's nothing else to work on, all the artists will be working on these areas, no matter how small they may be. There will be some polishing of the rest of the work, especially areas that don't blend together well, or where pieces don't fit quite right, but since the most skilled artists are working on the unfinished parts, only developing artists are available to do the polishing, and they don't want to wreck the project by trying any changes that are too drastic. Eventually the project end day comes. It may have slipped a few times during the project, but eventually it will come, and the project will be done.

There will hopefully be a few reports written about why the project was over budget, and why the integration took longer than anticipated. A few of the artists will quit or retire, and the most senior of the rest will be promoted to fill the holes in management. Then new artists will be hired to fill both teams up to identical sizes. Artists who were promoted to supervisor will take classes about how to design pieces since they've never been a part of the design team before. The other artists will take classes on basic art techniques to make sure that everyone has a minimal set of skills. Of course all tasks can only assume that minimal set of skills since no artist is ever taught anything above that. Eventually the team is ready to take on the next task, and the management team starts the next round of meetings...


Now which painting do you think is better? Which one has taught their team the most skills? Which one took longer? Would any company actually act this way?

There's no part of the analogy that can't be used to describe software engineers working on a big program. In fact the majority of companies to exactly the things described. There's also some truth that as the team gets larger, the collective design approach gets less and less credible. But a small solid team can be far more productive than a large bureaucratic organization.

So is there any hope that companies will realize the follies of their ways? Maybe a few, but I'm not holding my breath.

The AI Singularity

Since the idea of computers and robots was conceived by some geek in a cave people have been talking about the day that computers would be smarter than us. Some have written books about the utopia that will exist when the robots meet our every desire without us ever having to work again. Others talk of the day when the computers fight back and try to destroy the inefficient humans. But in all cases the critical moment comes when computers start thinking for themselves.

But what does it mean for a computer to "think"? All existing and future computers will always just run a set of instructions through a glorified calculator. You can add some randomness to make it seem less formulaic, but there's still a program underneath it somewhere.

In the early days of computing there were hopes that some really smart guy would write the AI program that made the computer "think". As time passed many nerds got many PhDs describing why it was so hard, but no one made anything that even slightly resembled thinking. Even today there are lots of engineers trying to make "neural networks" and "adaptive algorithms" that they hope will be the AI program that makes computers think.

So what will we do when we get this amazing program. Well, anything we currently spend brainpower trying to do, the harder the better. Interestingly, one of the hardest problems we look at today is making a better AI. What if a program could make a better version of itself? What would we do with that better version? We'd use it to make an even better version, of course.

This is where it gets scary... Today, our advances in AI are limited by the rate at which humans can improve their algorithms. Our tools (i.e. computers) get better all the time, and so does our state of research. But it still takes months or years to achieve notable improvements. What happens when a program can compile a better version of itself in a few days or a few hours?

Researchers call it The Singularity; the day our rate of research in AI improves by a few orders of magnitude. Suddenly our computers are getting dramatically better every day. A human goes to school for years to learn the basics of knowledge, even longer to become an expert. A computer program could replicate itself thousands of times in a day, and each would have the full expertise of the original.

Even tasks that are better suited for human thought than machine algorithms would quickly be switched to hardware when thousands of AI experts can work on a problem 24 hours a day for the cost of a computer and some electricity. Humans will never work again (or we'll be enslaved by the robots depending on your optimism). Our rate of scientific discovery will grow by orders of magnitude overnight... The world will never be the same...


So when will this first true AI get written? When is the singularity? Is this all hogwash? Well, it's certainly possible that making a real AI is harder than any of us can imagine, or it might even get proven impossible by some genius of the millennium. Then again, the program might already have been written, and is just in the process of getting itself into a good first working version.

Since the first computers people have been predicting AI is just around the corner, but I predict that the day is going to come soon. After all it's not that the computer needs to become a perfect thinker, it just has to get better at designing AI than us. We've had over 60 years with computers and we haven't achieved a half decent AI yet. That isn't such a high bar for the AI to beat...

Firefox vs. Internet Explorer

Ok, so with that intro out of the way, I figure I should post a real entry that is both technical, and won't scare away anyone who came over from The Tweedles. Let's talk about the new browsers that came out this week. There's a brand new Internet Explorer 7.0, and also a new Firefox 2.0. Now a lot of you are still IE folks, and if you haven't tried Firefox you should. It's easy enough to install; Just follow the link and you'll have a new browser running in about 5 minutes.

The main reason I would suggest switching is security. It's not that Firefox has less bugs (although it might). It's rather that hackers and spammers and all the other evils want to attack the largest crowd they can. And in browsers that means IE. It's like using a combination lock to secure your bike if everyone else uses a key lock. A thief is going to get really good at picking key locks so they can steal the most bikes, even if the combination lock is no harder to pick.

The second reason is that it's open source. A lot of you may not really understand what open source is and why you should care. The idea is that one group can write a program that achieves something, and everyone else can use that program to do anything they want, for free. The programmers gave their free time to make the product and they're happier the more people use it. The reason to support them is more philosophical than anything else. Do you wish all software was free? If so, support the open source community. Linux is the other big open source program most of you have heard about, but Linux is a little rough around the edges for most users. Firefox on the other hand is ready for general use, and anyone who can figure out IE should have not problem with it. It's the easiest way to show your support for free software.

The third reason is the new features. A lot of the features are in both the new IE and Firefox, but if you're going to upgrade anyway, you might as well combine it with the first two reasons and get Firefox. The one I want to focus on is RSS, Really Simple Syndication. It's almost as easy as the name would suggest. Once you've upgraded to Firefox, in the title bar beside http://thetweedles.net/tech there will be a little orange button that looks like sound coming from a point. Click it! It should ask you if you want to add a live bookmark. This means that when you look at this page in your bookmarks you'll be able to see from the bookmark menu all the posts I've made. Now you don't have to visit the site just to be disappointed that I've fallen in to a well and haven't posted in a while. Most of the blogs you read will support RSS, so start using it!

So upgrade you browser. Every time you visit a site it tells the site owner what browser you're using, so I'll know!!!

Hi Everyone

So what am I doing starting another blog? Don't me and my wife already have such a thing? Sure, but the people who read that thing aren't often the type of people who want to hear me rant about the state of modern programming languages, or the ways in which software engineers break the traditional labour model. These are the types of things that are often on my mind, so I figured rather than spit them at people who like Dea's wonderful blog, I'd try to start a following of my own.

Now if you're expecting me to tell you how my day went at work, you'd probably have better luck with the other blog. This will be mostly about things that I think of regularly. So wish me luck, and we'll see if anyone cares what I rant :)