My Personality

So yesterday I got to spend most of the day in a "Teamwork and Collaboration" class. Woo Hoo. It was 7 hours, and contained about 2 hours worth of information, but it came with free breakfast/lunch/snacks and I suppose that spending some time interacting with teammates is a good thing.

The main idea of the course was to identify everyone's "Myers-Briggs" personality type. Basically you determine which way you lean on 4 traits and that determines which of 16 types you are. Then you look in the book at features those types tend to have, and things they tend to not have. Then we looked at how different team members can bring different things to a group, but how we have to adjust our communication to bring everyone in.

Yay.

I'm not going to blow air at how I'd have rather spent the day coding than than talking about feel-good corporatisms all day, but I would like to talk about what the test had to say about me.

For those who aren't familiar, and don't care for Google, a summary of Myers-Briggs 4 traits are:
Extroverted or Introverted
Sensing or Intuition
Thinking or Feeling
Judging or Perceiving

Anyone who knows me or has read any of my blog can probably guess that I'm a Thinker. I'm also Extroverted which seems right seeing as I prefer working in groups. Perceiving vs. Judging is basically whether you like to make a decision and stick with it or leave the decision in the air and learn more about it. If you've read my blogs, you probably know I'm not a big fan of schedules and feature lists which makes me a perceiver. So far so good.

The second category was the one that I found most interesting. It's called the information category, and it is determined by how you get info about something. Some people like to gather up the details of the project and figure everything out from there. These are the Sensers, and they tend to be very good at detailed work, but tend to be bad at seeing the big picture. The Intuitives would rather start by figuring out the why of the problem before the how. The details aren't important except in how they come together for the bigger idea. Intuitives are very good at seeing how the project can grow and interact beyond itself.

Now I see a lot of myself in both sides of it. I'm a very meticulous programmer, and I'm very detail oriented when the importance of the work warrant it. On the other hand, I'm always looking at how a program will interact with other things, and I'm always designing code with future expansion in mind. I'm not inclined to start working on something until I understand how it interacts with everything else, and I if it's appropriate I would rather change the goals than blindly work to meet them.

So am I strong at both? Neither? Somewhere in the middle?

For what it's worth, the MB test gives a cute little name to all 16 types. The two I was torn between were the sensing Promoter (ESTP) and intuitive Inventor (ENTP). Based solely on the two titles I'd probably pick the later, but the description accompanying the former fit me very well: "Maverick non conformist who loves problem-solving and trouble shooting. Gets stuff done. Won't "go with the flow" if he doesn't agree with a decision. Tends to ignore how he will impact others emotionally". The Inventor description was good too, but much like a horoscope it's always possible to find a bit of yourself in all the types.

So what did the test rate me? On each trait (E/I, S/N, T/F, P/J) we got a score between 5 (even) and 10 (extreme). Most scores were 6 or 7.

I was an ESTP: 8, 9, 8, 6.
I'm a 9 for sensing!

How can that be? How can I be so good at big picture stuff? I was starting to think this whole day was going to be a horrible waste of time. How could the descriptions of sensing and intuition be opposites if I identified so well with both, and how could the test give me such an extreme score if I'm so balanced.


As the day went on we started talking a lot more about each of the traits, but I kept flipping through the book and slides to figure out why my information trait seemed to make no sense. The in-depth reading started to shed some light on the subject. The conclusion I reached:

I don't see the big picture, but I can see an enormous number of details.

I approach a large project as a large collection of details. I'm happy to sit around talking about what a program should do at a high level, but in my head I'm not happy with a suggestion until I can see how it would be implemented. Fortunately for the intuitives in the room I can "pseudo-mind-implement" most things very quickly. But I'm not really seeing the big picture outside the project. As more an more ideas are expressed as to what features should be added, I'm working overtime to pseudo-implement them all and to see how they fit with each other. This allows me to identify conflicting ideas very early as their pseudo-imps won't be compatible. It also lets me suggest features that would be easy to implement based on combinations of other features.

I basically build the whole program in my head. That's probably why I'm so quick at writing code. People find that my work style is to talk talk talk for ages, and then I'll implement the whole thing one night. They just see the code go from 0 to done in no time and think I could be amazingly productive if I spent more time coding and less time brainstorming. Alas that can't be...

So in the end it seems I really am an extreme senser. So what differentiates me, a big-thinking senser, from a real intuit? Well I think that I'm bad at the creative side of brainstorming. I'm great at assembling ideas to make new ones, but I need ideas to start with.

Need is the mother of invention, and I almost always find myself working on projects that need to get done. When I took some time off to be a ski bum, I did very little programming. I had all the time in the world, but no pressing projects. The few times I was working on something, I'd quickly process the whole task and spit it out, but then I was left waiting for more. Of course I have a few ideas in my mind about fun things to work on, but none of them sprung forth into a real project that actually got much done.

So I did learn the value of a diverse team during the 7 hours. I need the external ideas and projects so I have something to solve. Now I just need to get out of silly team-building exercises so I can get back to brainstorming with inuitives.

Fluid Labour

(Yes labour has a 'u'. I'm Canadian dammit! Without 'u', there can be no honour!)

Companies go to amazing ends when it comes to finding the right person for a job. They'll spend hours creating a job description, they'll post it in hundreds of spots. They'll take it to universities and trade conferences. They'll parse through thousands of resumes. Then there's the dozens of phone interviews. Then a handful of candidates are flown across the world to spend a whole day getting quizzed and questioned on their appropriateness for the job. They're asked about their skill set, their interests, and their previous experience. The top candidate gets a offer, and then negotiations begin to determine if the candidate can be hired for what he's worth. Sometimes the offer keeps going around trying to find the right candidate at the right price. If all goes well the new member gets a relocation package, a signing bonus, and sometimes an immigration lawyer.

The total cost of all this is tens of thousands of dollars for each position, so a small team will cost hundreds of thousands to bring together. Finding a stellar manager to lead this team is even harder, with head hunters, employee referrals, and even larger bonuses all increasing the cost.

In the end though, this well designed machine will be perfectly designed to finish the job at hand. Sure there's never a guarantee that they'll be able to succeed, but if they can't do it, no one can! ;)


So what do we do with this team after the project is a resounding success? Do we thank everyone for their time and effort and then make them cash out their stock options? Tell them to apply for a job on a new project?

Of course not. These people were expensive to hire, so we'd like to reuse them if possible. But how does Corporate America do that?

Most corps are horribly inefficient about allocating labour for their next project. Instead of building a team around a problem, an existing team is given a problem. This project is usually similar to the old one, but will always have differences. Some members of the team will have specialties that don't apply to the new work, while there may be a new need for specialties that no existing member has. The old manager may not no how to lead the new type of project.

The existing members have also gained experience, and want advanced and challenging work to reflect their new abilities. Corporations usually resolve this by promoting the top couple of employees to supervisors, and hiring some new people so they have enough people to fill out two teams. The management pyramid begins. The new hires are selected to be a good fit on the new project, but they are also selected to be inexperienced juniors because they'll be working under a young supervisor.

More often than not, one of these new teams gets the project: "Maintain the last project!" Fun! Granted, the people with the greatest knowledge about the project are exactly the people who made it, so this group contains qualified people. But most of them are over-qualified. The type of team necessary to build a project is usually far stronger than the team that maintains it. Companies are tempted to keep the most prolific programmers on the old project because they understand the most code. But of course it is these members who are the most over-qualified to stay on the support team.

For the other team, half the strength of the old team will now be gone to maintenance, and in their place will be a handful of juniors. The great old manager has also been lost to the bureaucracy and the team is now lead by the a new supervisor. How was this super selected? Usually he was promoted because we was the best programmer. His management skills may be non-existent, but they couldn't promote a lesser member without fear of upsetting the new super.

Management will now select a project for this team to work on. Read that again! We've gone from selecting the perfect team for a project, to finding a project for a team. The makeup of the team is determined by office politics, and the make-up is fixed even before the project is selected.

If a project suits a certain mix of employees, it is highly improbable that the new team will have all of those skills. Sometimes a new employee can be brought in, but obstacles such as unexperienced supervisors can make this difficult. In many ways it's even harder to bring a skilled employee from another group within the company than it is to hire one from outside. No team ever wants to give up a "headcount". You can't even trade for the employee because he's typically more senior than who you can offer.

The size of the team is also determined almost exclusively by office politics and irrelevantly to the project. Companies feel obliged to reward well done projects and seniority with new "headcount", and would never think of shrinking a team because it would send the opposite message.

Occasionally the teams and projects will get so out of touch with reality that the company does a "re-org". For the first time, the company considers firing people who are bad performers. Unfortunately they don't fire employees who have done good work in the past, but are not relevant to current projects. Large teams are built up under strong managers for big projects, and newer supervisors are given little teams and projects to grow under. Ideally this "re-org" would be the same as building a team from scratch, but far too often existing teams are largely kept together and moved around the bureaucracy. Most managers though will not lose significant "headcount". Instead they will take on multiple, unrelated projects that happen to have a total number of team members close to the old "headcount". Small teams get treated especially harshly as they can be easily "traded" between managers to balance "headcount" disparities.



Now I've certainly glossed over a lot of the reasons companies do what they do. They can't fire people without due cause or they'll get sued (unless they re-org, in which case they just pay a severance(?)). New college grads are initially quite useless but get put on teams so they'll learn and grow. And sometimes you've got a strong employee who you expect to need in the future and you want to keep them around so you toss them on someprojectoranother.

These reasons are valid, and some are due to regulations and social norms that are outside the control of the company. Many though are easily fixed by changing the corporate atmosphere. When a project completes, let the team dissolve, and let new projects find them. Tell people in advance that they will have to find a new position every product cycle. If someone has developed the skills to be a supervisor, let a project in need of those skills find them. Let a senior developer move on to a new exciting project. Form a new team to maintain the old project and try to recruit from the old team's members, but pick them according to needs, skills and willingness to work on it.

The hardest step for management: Some managers will end up with fewer "headcount", and some managers may not be able to find new projects. Which leads too...

And the real hardest step: If someone can only find positions the value him less than his current compensation, give him a pay cut. If no team needs someone at all, let them go.

This is easy for deadwood employees who aren't worth a thing, but it's incredibly hard to do for specialist employees, which includes some managers. A company might have spent a quarter million dollars looking for the right XYZ specialist, but if that project is done and no project needs XYZ right now, let them go. If you force them to do maintenance, or make them a supervisor, or any other waste of their talent, they'll just leave anyway.

This system doesn't lessen the value of specialists, but rather makes them incredibly important. Instead of running a project with the team you've got, you can build the team you need. You're free to hire a specialist from an existing project, or outside the company.



The opposite system to Corporate America is the start-up system and it too has its flaws. There's certainly a great flexibility to try to hire the best specialist for the position, but while there's a will, there aren't the means. A start-up simply can't afford to spend many tens of thousands of dollars plus the likely more valuable time required to hire these individuals. They also usually can't offer the same level of compensation, instead offering stock options (read: lottery tickets). Much of their flexibility is reduced because they can only attract a limited number of candidates and can only spend a limited amount of time reviewing them.


Ideally then, a system would exist where a company has large resources available to try to recruit individuals into the company, and then within the company there would be fluid labour system that allowed members to move to where they are needed.



On the surface Microsoft has a system that resembles this. They have hundreds of teams, and they "hire" internally, and when a team is dissolved the employees are given a month of paid internal job search time.

There are a number of crippling limitations however. The first is that teams usually only dissolve when a project is canceled. A project like Windows is never going to be canceled, so the normal bureaucracy of "headcount" and promotions ensues. Even teams making one-off products like video games are likely to move on to a sequel without really disbanding the team.

This bureaucracy then taints even the otherwise agile teams. Compensation becomes directed from the central bureaucracy so that people on different teams at the same "level" are given similar compensation. When a project starts up, they are allocated a certain budget (which makes sense) but they can't chose the salary to pay internal candidates, so you can't pay a premium to try to attract specialists.

The only way to reward hard work is by promoting employees to the next level, but the number of promotions available is a fixed percentage of the team size rather than being dependent on the quality of the team. This can cause employees to not get promotions they deserve. Then, having not been promoted, they aren't at a high enough "level" to apply for jobs for which they are now qualified.

This problem is worst in teams that require the highest quality engineers. Since these teams end up with so many great people, some inevitably get screwed out of promotions. These intelligent people then figure out that to game the promotions system they need to work on teams full of idiots so they stand out and can get lots of promotions quickly. But they tire of working with idiots (especially since idiots are usually hired on to less challenging projects). And eventually they quit, and the strongest teams are forced to reduce their quality in order to give qualified promotions.

If Microsoft opened up their compensation system to be as fluid as their labour system, they would likely find that the hardest projects would attract the best engineers for a premium. And if they worked harder to reduce the "headcount" and perma-team effects most groups have they might find that the make-up of teams becomes more suited to the projects.


Heck, I might even go back...

Surrounded by geeks

I've had the opportunity to live in two of the worlds greatest geek-meccas: Redmond Washington, and Silicon Valley California. Both of them are dominated by their high-tech employers and geeky inhabitants, and they are far more similar to each other than to most cities. But they also have some significant differences.

I first lived in Silicon Valley as an intern during the height of the Internet Boom. It was an insane place to be. Despite the fact that cities like San Fransisco and Oakland are a mere 45 minutes down the road, life here was different. The most obvious difference was the ratio of males to females. Santa Clara county managed to beat Anchorage for the worst ratio in the US. The result was that single women were non-existent. If a woman was attracted to rich geeks she then was taken by a geek with a lot of money, way more than you'll ever have. If a woman wasn't attracted to rich geeks she didn't spend any social time in the Valley because she would get hit on by 10 geeks an hour, one of which was insanely rich. I can't blame either group of women. If you like geeks there were too many to count so you might as limit yourself to the large selection of rich ones. And if you don't like geeks, you'd have to avoid them.

The money that was flying around here brought other oddities. There was the BMW effect whereby my boss drove a 3 series, his boss drove a 5 series, and his boss drove a 7 series. Around 30% of the cars in the tech company parking lots were BMW. There was a 4 month waiting list to by a new BMW, but they'd loan you an old BMW while you waited. Also, they only had a couple of cars on display, and when your car came in it went on display for a few days until the next one came in.

But the really special thing about the valley is how people you casually meet are always working at leading edge companies. I can't go to the dog park or climbing gym without meeting a dozen googlers. In fact I've found it's usually best to assume someone works at a high tech company until you know otherwise. In addition, everyone you know meets techies all day too, so any group of people knows hundreds of techies. The six degrees of freedom game is easy in the Valley. If you don't know someone at a particular company, someone you know probably does.

This makes lunchtime conversations very different from the standard water cooler dribble. Someone will mention some article they read, and they'll mention what site directed them to it. Then someone will say that they know someone who works at that site, so conversation moves there. Then a third person will mention how similar that is to another company he's heard about that is doing it a little different. And then the table agrees on a great idea for a new company. (And then nothing ever happens because ideas are cheap in the valley :) )

Having discussions like this everyday keeps the mind sharp, and keeps people up to date on technology. Not just websites, but programming languages, game consoles, and anything else with a chip is up for discussion. Yes it's a little geeky, but it does keep you thinking.


So how then was Redmond different? Like SV, you can assume that everyone there works for a tech company, and that they encounter nothing but techies every day. The difference is that there's really only one company: Microsoft. As much as MS has a monopoly on the desktop, its monopoly over Redmond is worse. We would joke about the social experience as a black hole where you could only ever meet people within the MS hole, things outside it were beyond the event horizon and could never influence you.

The day to day life is about the same, with lots of geekiness and lots of money. The weather in Redmond is far worse, so travel was a more common pastime, but things like nice cars and electronics were still very popular.

The social hole however meant that the lunch time chats were pretty much restricted to MS products. Granted MS is a huge company that encompasses most of the tech spectrum, but there was always a huge bias. It's not that Microsofties think MS has great products, or that Windows is the ultimate OS. Rather, they aren't forced to look at any alternatives. For every new website that grows popular, there's a MS clone site that works almost as well. And everyone in Redmond probably knows someone on that team. It's incredibly easy to forget that not every product is integrated with MS, and it's even easier to dismiss the competition.


Recently Redmond has started to diversify a bit, with Google in particular building a large campus nearby. This will hopefully help open up the social hole, and that can only be good for the microsofties and MS.

Silicon Valley on the other hand has had a marked decline since the heyday of boom, and it is now conceivable that it exists in the same state as regular cities.

Both places though are incredibly unique in their geekiness concentration, and that is certainly one of the reasons for their success. As a geek, I love the feeling of being "home", surrounded by my kind. But it is important to remember there's a real world out there that just doesn't have an opinion in the Perl/Python/Rails debate and would rather just have a program that doesn't crash so much :)

Map of top 100 technology companies in the world

Have you ever wondered where most technology companies are? I did. I've seen maps of the biggest 10, or the biggest 10 in silicon valley, but I wanted a bigger overview. I also was looking for something fun to do with the Google maps API so I put something together:

Map of largest 100 technology companies by market capitalization

I got the market cap and address information from Google finance sites. I wrote a quick program to run through the pages and grab the info. Alas there was a lot of manual scrubbing to clean it up, but it's done now.

It appears that about a third of the companies are based in Silicon Valley, although of course all of the companies have some presence here. The majority of the companies are US based which is also interesting. Each dot also shows the map cap (in $'000) and the rank. Poor lonely MS is number 1, but it's over 1000 miles from the next closest mega-tech-company.

Anyway, I hope you enjoy.

Programming and Professionalism

Software Engineers are known as a rather slack group of individuals when it comes to traditional "professionalism". We aren't a 9 to 5 type of people, and we don't appreciate suits. Many of us would rather have an interesting job than an powerful title, and we idolize the start-up of 3 guys over the CEO of 10,000.

So why aren't we as professional as others, or perhaps the better question is why are others so much more professional than us?

A simplified look at the question occurs during job interviews. In many ways an interview is when we act the most professional. We show up 10 minutes early, we wear a freshly dry cleaned suit, and we act in an absolutely politically correct manner. Even if our natural self would roll into work at 11:30 wearing track pants and a t-shirt, we feel obliged to act differently during the interview.

The interview is itself usually a horribly mal-suited way of hiring people. After a few hours of interviews:
What we want to know:
1. Are they a good worker?
2. Will they fit into the company?
3. Will they be loyal to the company?
4. How much are they worth?
5. Do they have the necessary knowledge for the position? (Do they know how to program?)
What we actually know:
1. Are they comfortable under pressure?
2. Can they answer quasi-philosophical questions with reasonable BS? (What's your greatest weakness?)
3. Can they quickly solve very simple programming questions on paper?
4. Did the candidate know anything about our company?
5. Did they show up on time, wear a suit, and avoid saying bad things?

So how can an interviewer determine what they want to know from what they've found out? Well, often a candidate will show something during the interview so you can determine they aren't qualified, or otherwise won't be a good employee. But most people know the game, and even prepare for it much like they did finals in school. They prepare an answer for "their greatest weakness" type questions, and they review a bunch of simple programming questions so they'll look competent. They'll do a little research into the position and the company. Hopefully they'll practice controlling their nerves, and finally they'll make sure to show up on time, in a suit, and they won't say anything controversial.

So the most obvious thing an interview really tells an employer is "Did this candidate prepare for this interview?" Now while this isn't one of the things we hoped to learn, it is useful. A candidate who successfully prepared for an interview is likely to prepare themselves for projects at work. More importantly, a candidate who didn't adequately prepare for the interview is unlikely to adequately prepare for projects at work.

So the real point of wearing the suit isn't to look good, but rather to show the interviewer that you know and understand that you're supposed to wear a suit. Similarly with showing up a few minutes early, and keeping conversation very sanitary. It's all a game, and the interviewer wants to see that you are capable of playing. The game itself is somewhat arbitrary, but you still must follow it in order to show you understand it.

Attire is also very important because the interviewer sees it immediately before they ask any questions. If a candidate shows the lack of ability to "play the interview attire game" it will taint the interviewers opinion which can negatively affect the whole interview.

Now there's something I've glossed over. In software interviews people don't usually wear suits. Instead the candidate will wear khakis and a collared shirt, no tie. So does that mean the potential software candidate is not playing the game right? Quite to the contrary, he's showing that he understands the subtleties of the game. He knows that the interviewer has seen many candidates without suits, and that the interviewer will not be wearing anything formal either. The business casual look still shows that the candidate put thought into his attire, but also shows that he knows software is different from other jobs. The candidate who shows up in a suit looks out of place and distant, and is almost as bad as the candidate who shows up under-dressed.


So that takes us back to the original question: Why is software less formal than other industries? Well, it may be that wearing professional attire is simply showing to others that you can "play the game". When a salesperson meets a customer, it is much like a job interview: The customer is trying to determine whether the company's product is good or not, but cannot in the limited time possibly make a full assessment. So instead they rely on things like the salespersons professionalism. It's just an arbitrary game, but a salesperson who doesn't play the game seems to not understand what is expected of them, and therefore seems out of place.

Many jobs follow this line of thought. Anyone who has to deal with customers, or other companies, or even other departments depends on making a good first impression. It's not that the way they look is important to the quality of their product, but the other party doesn't have time to fully evaluate their company, so they evaluate how well it's representatives "play the professionalism game".

Software then plays by very different rules. We typically don't interact in person directly with our customers. We make the software, and then our marketing and sales departments try to sell it. Requests come back to us, but they're filtered through our support department. These other departments all act very professionally of course, but when we interact with them it is usually through email or phones. Also, we know that we are making the product and that therefore we are the center of the company. If we upset someone in sales it's a lot easier to replace them than to replace us.

Some companies such as Microsoft have taken this separation to the extreme. MS does development in Redmond and Silicon Valley, but all Marketing and Sales is done at different sites like Dallas and New York. When I worked there (yes I was a Microsoftie) I never once spoke to a person in Marketing or Sales. The isolation meant that working hours and working attire were extremely flexible. It was still important to respect other Microsofties, but it was understood that wearing your pajamas to a meeting wasn't meant to be disrespectful.

Some companies go even further with the separation. Typically a start-up will not have any customers for it's first few months. Most of it's future clients won't even know it exists. This lets the start-up work whenever, wherever and however they want without upsetting anyone.

This in-formalness is of course very positive to the creative process. Since programming is so creative it is important not to stifle it. Of course eventually the start-up ships their product, and now they have customers who expect someone to help them with their issues during normal business hours. They expect the people they interact with to be professional, since they don't have the time or patience to evaluate the company on its real fundamentals.

Separating the professional company facade from the creative programming engine is likely to remain a great challenge in the software industry for years to come...


-----

Paul Graham has some great essays of his own on similar subjects:
What Business Can Learn from Open Source

Why Nerds are Unpopular

If you like my writing, I highly recommend his essays.