Posts tagged "software-design" - nolan caudill's internet house

Three Levels of Developer Productivity

Fri, 11 Feb 2022 22:28:31 +0000

If you work on developer productivity, you sit in an unique spot. You are often given a broad mandate — “make the engineers more productive”-and typically little guidance on what the word ‘productive’ means. (Spoiler: There’s no one right answer here.)

To start to give some shape to the job, in my experience, there are two important categories of problems you are solving. There are other categories, of course, but these two feature prominently and will drive much of the work.

The first problem is one of engineering morale. The tools engineers use for their work factor greatly into the satisfaction they get from their jobs. If you place too high of a hurdle between an engineer making a change and them seeing the impact of that change, or if you make the risk of making those changes too great, you will sap people’s satisfaction. People do quit over slow build times and people do quit over dangerous deploys.

There’s also a breadth of research that show that teams that have fast feedback loops are higher performing. While this research may speak to many, what I’ve found to be universally resonant is the ethos of “don’t frustrate engineers with bad tools and processes.” Nearly everyone can get behind that.

The second problem, in coarse terms, is engineering return on investment. There is a good chance that one of your company’s biggest expenditures is engineering headcount. And every new feature shipped and every bug fixed is generally gated by an engineer doing something. Therefore, engineering cycles are one of the most debated, negotiated and protected resource in any tech company and the bulk of processes have their roots growing out from the perceived scarcity of those engineering cycles. There will always be a natural pressure, from all angles of the business, to get more from less, to speed engineering up, to help engineering do “more” and whether said directly or not, part of your job is to understand this and make improvements here.

If you’ve been around larger organizations, you’ll quickly see that some of these productivity problems can be solved with engineering elbow grease, but it’s not hard to see that engineering as a whole is often slowed down by the sheer coordination overhead that comes naturally to complex organizations running complex technical systems.

The challenge is that improving build times will make a small dent if any on coordination costs (unless you are starting on one extreme and moving it another extreme), leaving the “engineering is the bottleneck” problem largely unaddressed. But, when you start to focus on the second problem, the work isn’t as easily understandable to others.

To make the work more understandable, a framework I’ve had some success with is to explicitly put developer productivity efforts under 1 of 3 different banners: individual productivity, team productivityandteam-of-teams productivity.

Individual productivity: In my experience, this category of work is what is typically seen as the core duty of developer productivity teams. This work includes efforts like speeding up build times, reducing test flakiness, increasing deploy cadence, making debugging easier and so on. The primary focus is on the individual developer and moving the code they author through the pipeline faster and more reliably.

There is tremendous value in this category. If you can take a routine process that every engineer is involved in multiple times per day and speed it up by 10%, that’s a huge number of valuable engineering cycles that are freed up, making a scarce resource that much less scarce, speeding up valuable feedback loops and making engineers enjoy their jobs a bit more.

And, at some point, some of these processes hit a phase change where because something has become so cheap to do, it encourages a whole lot more of that thing to happen. Think of the difference between a full build taking a day and that same build taking 10 minutes, or a weekly deploy becoming on-demand. The entire economics of your engineering processes change and it has knock-on impact across the business.

**Team productivity:**This category focuses on improving production by focusing on the flow of work between engineers, and usually engineers within the same team.

This is the realm of team processes: how does code get reviewed, how many projects can one team undertake, how does a backlog get managed and so on. This is where words like “agile” and “scrum” get mentioned.

In my experience, individual teams often have a lot of leeway in how they manage themselves. Where bad tales arise is often from a top-down mandate to “become agile.” There is also value here but it is riskier and efforts here will falter unless the most senior leadership is actively involved, you have a network of coaches throughout the org and you are willing to take a short term (eg, 3 to 6 months) dip in productivity while teams learn to operate differently and tweak their processes. Lining up and sustaining these preconditions is tough and there is no one weird trick to pursue here.

One approach to this level of productivity is to focus on the handful of teams that are recognized as a shared resource (eg, SRE, centralized tools teams, platform teams) and are on the critical path for some key projects and then help improve how they take in, prioritize and execute their work. You’ll likely get a large percentage of the value of “becoming agile” mandate while not actually having to spend the time and effort to make every team implement Scrum, as an example. Your org likely has a few critical teams that are the bottlenecks or dependencies on many efforts and there is high leverage in improving those specific teams’ processes.

**Team-of-teams productivity:**Problems here possess the delightfully confounding combination of being harder to see, requiring a delicate touch, are usually seen as “not your job” and are notoriously hard to measure. Yet, in my opinion and experience, making headway on these problems is where real leverage lies, not just for the engineers but for the whole business.

This level is the realm of quarterly/yearly planning processes, org design, platform migrations and other processes that are expensive, hard to change and slow. Simply because these things are expensive, hard to change and slow make any improvements in efficiency here incredibly valuable.

One example: Migrations are a fact of life of larger and older orgs. At any point, there are likely many migrations any individual team could be involved in, or that the organization would like to be doing. This presents a tradeoff: is it better to focus on one migration at a time, or to spread your efforts across many migrations? In general, migrations have little value mid-migration — you have the overhead of living in two worlds simultaneously plus the overhead of the migration itself. So reducing in-progress time should be a priority. This leads naturally to focusing teams on a single migration, which reduces overhead, reduces technical complexity, reduces status checks and reduces context switches, which are all good things.

So in that example, a team-of-teams productivity improvement would be to cap the number of in-flight migrations occurring at any one time.

The ways to make improvements at this level is by having relationships with pivotal engineers and managers, getting involved at the right time (eg, quarterly or yearly planning) and making a case for a productivity-improvement constraint when stakes are fairly high and involve influential people with differing goals and motivations. Nudges and tweaks to existing processes typically work much better than large campaigns.

When the “CI/CD person” shows up to suggest improvements to quarterly planning processes, using the framing of improving team-of-teams productivity, with a focus on overall program and organizational effectiveness to help the whole organization deliver more value sooner, helps explain the motivation behind the suggestions.

— — -

Working on developer productivity can be hugely rewarding and there is a lot of value that comes from working on it at different levels, even outside the typical domains of automation and CI/CD.

Moderation Amplification

Fri, 20 Apr 2012 07:00:00 +0000

A few nights ago, I came across Tom Coates’ post titled Social whitelisting with OpenID… about how to handle moderating an online forum when the amount that needs to be moderated outweighs any one person or small group of administrators’ capabilities (due to time, sanity, etc). This post was written in early 2007 but this system which advocates building a web of trust between your friends was never built as far as I know.

It did remind me though of a moderation tool that me and couple other tech folks designed and built right around this exact same time for a TV station’s website overhaul.

One golden rule of the web is “don’t read the comments”, because, as nearly an absolute rule it brings out the worst in the worst people. On a news site, this is triple the case. In polite company, religion, politics, and money are things to tread carefully around even with close friends, but with the veil of anonymity that the Internet provides combined with a website that deals daily in stories of religion, politics, and money that often directly impact the reader, it’s a damned near-perfect recipe to attract explosive, hateful, and irrational comments.

As part of this TV site’s redesign, there was to be more of focus on contributions from the viewers, which were in the form of photo and video submissions, guest blogging, a simple “wall” for members to leave comments for one another, and, of course, comments on news articles.

As a hard rule, beside every piece of member-added content, we always put a link where anyone could report abusive content. This would go into a moderation queue in our administrative tools and our editors could act on it, either marking it as abusive or marking it as “seen but okay”. For a news site, with millions of visitors a day, this system wasn’t manageable, as there would be as many false positives and there would be valid abuse. For some, it was abusive if someone disagreed with them and they couldn’t find a way to logically defend their argument. This was overwhelming for a tiny editorial staff.

So, we had to devise something to at least bubble up the true offenders in the moderation queue. (Now, this was 5-6 long years ago and I’m sure it’s evolved since I left, but here’s how I vaguely remember it working.) The idea that we came up with was this: a member reporting abuse is right if we agree with their judgment, and people that report abuse “correctly” more often build up a certain amount of us trusting their judgment. If someone reported abuse 100 times and a 100 times we agree with them, there’s a really good chance that their next report will be correct as well.

So we assigned every user a starting trust score and for every time they reported abuse that we deemed valid, we’d bump up their trust score. On every abuse report, we’d look at the trust score of the person that reported it and if it met some threshold, we’d silently remove it from the site. Their abuse report would still exist in the system, but there was less of a time pressure to go through the abuse queue as after awhile, a small army of reliable reporters would be moderating the site for us.

On the flip side, if an abuse report was deemed wrong, their score would be drastically reduced, halved if I remember correctly. We were fine with the penalty being so severe as good users would build themselves back up and introducing a little chaos into this system was nice as different editors would have slightly different standards, and a lot of the time judging whether something was abuse or not was a judgment call. Chaos was inherent in the system from the start.

These scores were obviously secret, shown only to the editors, and I honestly can’t remember if we were actually doing the silent removals by the time I left, but I do think those reports at least got priority in the moderation queue and when going through thousands of reports, this was incredibly helpful.

I like to view this kind of system as sort of an ad-hoc Bayesian filter where your moderation efforts are amplified, rewarding and ultimately giving some weight to people that moderate like you do.

So, the social whitelist begins with allowing a subset of users to post, while the trust score model involves dynamically building a list of good moderators that agree with you on what is abusive content.

I still love the idea of social whitelisting or building up a set of trusted users to help you with moderating, as both are more organic approaches to moderating, meaning that it’s forcing you as a someone in charge of a community to actually make decisions about what kind of discourse you want on your site.

This is also why it saddens me a bit today as more and more blogs are just dropping in web-wide, generic commenting systems, like Facebook’s. While it is enabling almost everyone to be able to quickly log in and start adding comments, it’s horrible for the site owners that are trying to build an intimate community. Every decent community probably has a baseline standard of what’s acceptable: no hate speech, no physical threats, no illegal content, etc. This is what Facebook provides – a baseline – and nothing more.

Any community worth moderating is nuanced, has a voice and a direction. Facebook doesn’t offer this, so every blogger that drops in this commenting system is making that trade-off between ease of user engagement and being able to effectively manage a community. I’d like to see more sites go back to these more hands-on and thinking-hard approaches to how to moderate and direct their communities instead of relying on someone else’s standards of what constitutes a good contribution.

Thoughts on Pagination

Sat, 24 Mar 2012 07:00:00 +0000

A common navigation pattern on websites is what I call “chunked pagination”: each page has a predetermined number of pieces of content on them. Page 1 shows items 1-10, page 2 shows 11-20, and so on to the end of the stream. Easy. This pattern, though incredibly common, isn’t useful for navigation most of the time. The major issue is that it gives few hints about where those links will drop you in the stream. It’s an arbitrary chunking of how you actually display the content. Pagination should provide accurate navigation points that reflect the overall ordering of the stream, and pagination based around fixed-length pages provide nothing more than arbitrary access into this ordering, where we have to use estimation and instinct about the distribution of the content in order to make a guess of where a link will send us. Having a pagination scheme that closely models how a stream is sorted can give you both the casual browsing experience that the numbered pagination provides, as well as powerful navigation abilities that the numbered pagination can’t provide.

For example, take your average photo site that displays the content in a reverse chronological order: that is, newest to oldest. Let’s say your friend has posted 2000 photos to this site. The site shows the viewer 10 things per page. With our prolific user, this gives us 200 pages. Going to the middle of this content takes us to page 100. What does this mean, beyond we’re at the middle? Not much. Let’s say this user posted their first photo to the site years ago, but has just gotten back from a month-long trek through Europe where they took a thousand pictures. So our page numbers monotonically march back 10 by 10, but as we know this stream is sorted by date, and we want to go back in time on their photostream to a dinner you shared 6 months ago, we’ll just have to guess which page to start with. Since we know our friend’s usual posting velocity, we think that ten pages should take us a few months back in time, so we go to page 10. On page 10, our friend is in Europe, looking at the River Seine, just 1 week ago. Let’s go back 10 more pages. Hm, our friend is still in Europe, admiring the beach at the French Riviera. This is frustrating, so let’s try 40 more pages. Click. Damn, our friend is still in Europe (good for him, but bad for your navigation). After some clicking, we’ve got them figured out. We know that page 100 is right before their trip started, so our estimation applies again, disregarding the first 100 pages, we quickly find the dinner pictures. But, now that our friend is back from their trip, they resume their normal posting volume of 2 or 3 things per week. So, after our friend is back from his trip for a month or two, the first two pages cover a few weeks, while the next 100 pages cover four weeks. The concept of “page 100” no longer means anything, as this link is very much a moving target as things keep getting posted to the beginning of the stream. Lots of problems with the page numbers, it seems. First, we have to guess at our friend’s posting volume and frequency to even make a stab in the dark of how a particular page number relates to a point in time. Then, in an ideal world where URLs are meaningful, the page numbered link (to use Flickr as an example, http://www.flickr.com/photos/nolancaudill/page7/ is the seventh page of my photostream) doesn’t point to any specific resource, beyond that it points roughly to the 126th through the 142nd photos I’ve posted and will point to different photos on my next upload. This link is only “valid” as long as the dataset doesn’t change (which datasets tend to do).

Let’s look at the two main ways that people navigate a sorted list. The first, as we discussed, is by seeking. You know where something happened in a particular sorting (by date, in this example) and you want to get to it. Page numbers do let us narrow it down, but it’s usually a guessing game. Go too far back and you have to click back. Not far enough? Click further. Repeat these steps until you narrow down onto the correct page. The second way people navigate is by browsing, where there’s not a specific goal in mind beyond seeing some stuff. For this method, the page numbers are also not necessary. You either want to view the next page, or just jump to some point further in the list. For the former, a simple “next” link is adequate, and for the latter, you can provide this same action but in a way that makes sense for this case, but also for the “seekers”. The way to represent this that satisfies both the seekers and the browsers is to have pagination that actually chunks based around how the list is sorted. Some examples of this in real-life: dictionaries with the tabs on the page edges that show the alphabet, calendars (one page per month), and encyclopedia sets that have one (or more) books per letter. Dictionaries are actually a good representation of the problem of uneven distribution of the content, as the ‘T’ and ‘I’ sections are much thicker than the ‘X’ and ‘Z’ sections. Jumping halfway into a dictionary doesn’t mean much at all, but having those handy tabs on the edges give you a good head start. Also, even if lots of words are added or removed, jumping to the ‘J’ tab will always take you to the first ‘J’ word, regardless of changes in the vocabulary. On the web, blogs usually have both the seeking and browsing navigation controls. Wordpress, for example, has the ’next’ and ‘previous’ links at the bottom of the main list views, but usually provides a separate archive page that lists all the posts split out by month. Aaron Cope’s parallel-flickr has an interface that shows all photos uploaded by your friends in the last day. Instead of using pages, the list is divided up by the uploader (signified by their avatars), which is helpful as I have some friends that post one photo at a time, and others that empty their entire memory card in one fell swoop, but I can successfully navigate both cases. In all these cases, form and function work together nicely, with the pagination links reflecting how the underlying data is actually laid out, making both seekers and browsers happy. It also creates useful links such as linking to “January 2010” in a reverse-date ordered photostream will always be constant, regardless of how the data around it changes. Since it came up on the Twitters, I should mention the concept of infinite scrolling as a pagination scheme. (Since blech has a protected Twitter account, I don’t want to write out his tweet verbatim, but I can summarize it for sake of context by saying he took issue with an instance of infinite scrolling, which can be easily deduced from my reply). Infinite scrolling is basically a pretty representation of the ’next’ link that you ‘click’ by scrolling to the bottom of a page. I’ll leave whether or not it’s good user experience to others, but as a purely visual experience, I like it. If it’s the only source of pagination, that sucks, and another navigation scheme should be provided if having your users be able to look through the list or find something is important.

So to wrap this up, how would one create pagination links for our reverse-date order example, which is an incredibly common view? The obvious way is by actually chunking around dates. I believe people that are much better at designing useful things than myself could adapt this into the same form that our current paginate-by-arbitrary-chunk format occupies. I think adapting the links you usually see in an archive view could be represented in a succint form that makes both seeking and browsing easy operations.

Fewer Features Is A Feature

Sat, 12 Nov 2011 08:00:00 +0000

I saw where Linux Mint released their newest version today and one of their changes was switching the default search engine to DuckDuckGo. The little blurb describing DuckDuckGo was this:

It doesn’t show different results depending on who’s making the search, it doesn’t track or record user information, it provides you with optmized results and it’s built on and contributes to Open Source.

The engineer side of my brain immediately said, “Well, not doing per-user search results and not tracking users is EASIER to build! That’s awesome that he’s getting away with building less and tauting that as a good thing.”

People usually positively correlate the number of things you can click and look at with how “good” a product is. Microsoft traditionally has always had a problem with this, and a lot of other enterprise software does this as well, usually so they can just check off a feature request checkbox, with not considering how these features impact why people use the product in the first place.

One piece of software I use where they are proud of, or at least point a finger to, the features they don’t have is Pinboard, where it has routinely called itself the “anti-social bookmarking service”, due to its lack of a full-fledged social network in the Facebook sense of the word.

The current feature set of Pinboard is exactly what I want. It keeps my bookmarks, lets me see a few friend’s bookmarks with no automated discovery so there’s not the add-everyone-you-ever-met problem like most other social networking sites, and then just gets out of the way. I don’t need anything else and anymore things to click and look at would negatively impact the things I do need it for.

I’m sure these social features are easily in the realm of Maciej’s technical ability, but developing a social anything, involves things likes privacy levels, explore of some sort, content and comment recommendations, and each of these has its own scaling problems, not just in hardware and software, but in community support and abuse prevention. I’d rather Maciej’s time be spent making the current features better and attracting new users so he can make a good living and keep the service running.

Not having Feature X that every other product in your class has could absolutely be a defining feature. If a feature is not in your wheelhouse, for every user that finds it useful, there’s at least one other user that finds it bothersome, or at least in the way of using your product’s main focus.

I wish more of the services and products I used would be more strict about what they build and apply laser-like focus on that what their product is. I’d rather use many, small sharp tools than a few, large dull ones.

The Summer of Radiators and Incidental Complexity

Sun, 23 Oct 2011 07:00:00 +0000

During the summer between high school and college, a very important summer in most people’s lives, I worked as a radiator mechanic. The guys I hung out with in high school talked mostly about two things: girls and cars, and the latter was the only thing they could speak of from any experience. Me? I knew nothing about girls and even less about cars. At the start of the summer, I worked at a lumber yard, but when they cut hours down to just a handful a week for the summer crew, my uncle offered me a job to help out at his radiator shop and as a kid that needed gas money, I accepted, even though I didn’t know the difference between a radiator cap and a hole in the wall. There were three of us in the shop: my uncle Wayne, Terry, and me. Terry told me that God helped him find the money to buy the red 1969 Gran Torino he drove to work every day. With a car that nice, I believed him.

Every customer’s issue was a puzzle and I loved solving these puzzles. People would come in with a complaint, usually that their car was running too hot or too cold. From these high-level complaints, we’d start digging down, armed with the knowledge of how the fluid, heat, and metal played off each other. Car running too hot? Make sure there was enough fluid. If the coolant looked ugly brown instead of neon green, there was probably chemical build-up in the radiator core. If the fluid looked okay, the thermostat was probably busted since it wasn’t opening up and letting the hot water into the radiator to cool. That’s how every job worked. If this, then that. Just like the machines themselves, diagnosing them became mechanical.

The heating-and-cooling system of every car is basically the same. They all have radiators, thermostats, water pumps, and heater cores. A fresh-off-the-lot Honda’s radiator works the same way as the radiator in a 50-year old Chevy truck. Everything else around this simple system has changed, though. Someone would bring in a newer-model Honda Civic and want their heater core replaced. (The heater core is a mini-radiator that sits behind your dashboard that blows hot air when you turn the heat on.) We would quote a full day’s labor to do this. This same job on any vehicle before the 1980s would be more in the range of an hour’s worth of work. Honda Civics were built to be small, light, safe, and environmentally friendly. These are all good things but the complexity these new features added made the other simpler systems more expensive to maintain. In this case, since these things were built so compactly, you’d have to remove the entire dashboard of the car, including rolling up the carpet and removing pieces of the door, keeping track of dozens of little screws, panels, and trim pieces, just to get to the heater core. In older trucks, built with neither weight nor size considerations, the heater core was usually behind a little panel under the passenger side dashboard that involved removing four screws, popping the heater core out, putting the new one in, and then replacing the panel. Easy. Some of the jobs on newer cars would actually be a significant portion of the total value of the car. If it costs you several hundred dollars to replace a fifty dollar part on a car that’s only worth a couple thousand dollars, people often decide that’s it’s just cheaper in the long run to just buy a whole new car.

Even though the basic concept of moving water through an engine to control temperature has held across a 100 years of automobiles, cars in the past 15 years have made working these jobs longer, more cumbersome, and thus more expensive. Like a lot of pieces of modern culture, things seem to be built to be replaced, and not to fix. Some people may attribute this to our “throw-away culture”, but I think that it’s more of a symptom, and not the cause. In the cause of advancement, we’ve built things that are cheaper to replace than to fix.

This relates perfectly to building large software systems. You often hear tales of the Big Rewrite: when a group decides that in order for a piece of software to be improved, it needs to be rewritten from scratch. Their problems don’t exist at the component level. Writing to a database probably works fine. Displaying a web page probably works fine. Uploading a photo and storing it away probably works fine. But when these things need to exist alongside one another, software people tend to make them unnecessarily depend on each other. And when things are tightly coupled, you can’t change one without affecting other seemingly unrelated things, and not without great cost.

Complexity of one system affects complexity of the overall system in often unforeseen ways and most complex systems are made up of simple, but overlapping, systems. I’m sometimes nostalgic for when I was debugging and fixing a physical machine, since dealing with concrete matter made thinking through these complex systems easier. But when you’re dealing with such an abstract thing as code, it’s easy to write yourself in a corner, where you sometimes have to remove the whole dashboard just to replace a cheap part.

The Fear

Mon, 17 Oct 2011 07:00:00 +0000

I wrestled in high school. Our team was good, even winning a state championship one year, and our team was filled with several highly-ranked wrestlers.

By my second year of high school, I was a good wrestler. I’d go up against these ranked wrestlers every day in practice and at first I held my own, then I occasionally beat them, and by the end of the season, I’d win almost every time. My coach made me wrestle heavier guys in practice to challenge me, and I got to a point where I wasn’t getting scored on.

Once, I had an Olympic medalist at a camp use me as an example of how to do several moves. Of the hundreds of matches wrestled at another camp, I recorded the fastest pin. My coach told my dad that he would be surprised if I didn’t win at least two state championships before I graduated.

In practice, I was dominant.

That year, in official matches, my record was 7 wins to 27 losses. I lost and lost often and it usually wasn’t close.

At the same time, my teammates that I beat consistently in practice were all on big win streaks, winning medals, and qualifying for the state tournament, while I was lucky to win at all.

Through the rest of high school, I won more than I lost in official matches, though not by much, but in practice I became even more dominant, routinely beating everyone.

The difference was, before real matches, my analytical brain would think of every single tangential thing that had nothing do with the physical act of wrestling. I would think about my opponent’s record, how many points my team needed from me, and my past failures. During practice, when there was nothing to lose, the only thing that mattered was the wrestling.

When it counted, it was no wonder that I lost. I would psych myself out with pessimism and by the time I actually stepped out on to the mat, I was in a full-on nervous breakdown.

Looking back, I realize that I would have been fertile ground for even the most basic of sports psychology. When it didn’t count, I was a world-beater. Turn on a clock and throw in a referee and I became a completely different athlete.

Going ahead about a dozen years, I’m working at Flickr and a big piece of our developer culture is something called the Fear. This Fear is that fluttery feeling you get in your gut when you’re about to deploy one of the world’s most-loved websites and your changes could Mess Things Up.

The Fear comes from not having much of a cover-your-ass process. Developers get hired with the ideal that they are their own steward of quality. They can deploy code whenever they like, as often as they like, with little fanfare and with this privilege comes the trust that you’re not going to mess up (or at least not too bad, or too often).

I’ve worked with developers that don’t seem to have much in the way of the Fear, or at least didn’t show it. When I first started at Flickr, I was in the opposite camp. I had to force myself to deploy “big” things. I would get that familiar feeling that I used to get on the wrestling mat, paralyzed by analyzing every possible way things could go wrong.

On my desk, I have a print-out of the quote, “What would you do if you weren’t afraid of failure?” Turns out, quite a bit.

Having the hindsight of a failed high-school sports career with a healthy respect of my own limits, I’ve (mostly) worked through this by embracing a few core tenets.

The first is knowing that people mess up. I mess up. Constantly comparing myself against some invisible measuring stick of perfection is a surefire way to cause disappointment because, surprise, perfection doesn’t exist in the real world. Good enough does and going after that is, well, good enough.

Another thing is knowing that I can only do my best. When I write code, past history shows that it usually turns out working fine. I take this as my best has been good enough in the past and it will probably be okay this time too. I have to remind myself of that.

The biggest mental adjustment I’ve made is just trying to remove the emotion of fear from the deploy itself by approaching it analytically. I ask myself, basically, “Is this code good for deploy? Has it been tested? Does it work for you and the test cases you thought of?” If I can answer in the affirmative, I override my animal instinct that is beholden to the Fear, the one that is throwing out nothing but pessimistic what-ifs, and hit the button.

In the end, the only thing that matters is the code, and not any squishy human emotions that my own neurosis attaches to it. If the code works, I ship it.

If I had this same mantra when I was a wrestler, by just getting out there and doing nothing but focus on the physical act, I could have been a contender. But now, every day, I get to redeem myself bit by bit.

Photos:

Shipping Software

Wed, 15 Jul 2009 07:00:00 +0000

Shipping ordinary software on time is damned hard. Shipping great software in any time frame is extraordinary. Shipping great software on time is the rarest of earthly delights. – Jim McCarthy, Dynamics of Software Development

I received my copy of Dynamics of Software Development by Jim McCarthy in the mail today, and this quote was just sitting there on the first page.

I’ve shipped great software on time and I’ve shipped ordinary software late. This inconsistency to a logic-driven mind is maddening. I think I’m fairly decent at writing code, but getting the whole process to a point of repeatable success from a business standpoint is what I’m in the dark on.

Though my computer science education was fantastic in teaching me about logic and theory, the nuts and bolts of software engineering is something I am lacking. There was a software engineering course offered but I opted for the programming language design course, instead. I don’t regret that decision–I love programming languages–but in the world of waiting customers and deadlines that come much too quickly, I’m having to play catch up.

So for now, it’s time to hit the books.

Truth about Software

Thu, 25 Jun 2009 07:00:00 +0000

I spend a lot of my time thinking about software develoment, keeping up with what works for other people and what works for me. I’ve recently read Peopleware and Code Complete, and I’ve formulated some thoughts about software development and technical management that I’ve been meaning to write about. Hopefully, this copy-and-paste will get me going with that.

I found this printed out on Hacker News, and I think there is a lot of truth in it about the perceptions of software and its development.

The problem with IT, generally, is lack of respect. Because it’s something that looks easy and looks like something anyone could do, people don’t respect experience and talent.

For some reason this just doesn’t work in other fields. While everyone feels they can put a sticking plaster on a cut, they go to a doctor for more serious stuff. Any fool can build a garden wall, but if they want a tower block one hires an architect and a construction company and lets them do decisions.

Whereas, it’s quite common in IT for micromanagers to overrule the professionals.

It doesn’t happen in building. Managers simply aren’t allowed to say things like “Oh, I don’t think we should RSJs in those load bearing walls”. If the architect says they’re needed, then they go in. We also don’t hire doctors by picking the tools they’ll use. When you want a doctor to take out an appendix, you want a qualified general surgeon. You don’t ask them whether they prefer a #3 or a #4 handle on their scalpels.

And yet it’s pretty common to see projects which have picked a technology first (“Oh we’re going to use J2EE for this”) and THEN hire the team around that (“Wanted, tech arch, 5 years exp with J2EE, C , C++, JAVA, Perforce, Apache, Perl, Python.”) and only at that point produce the spec. And then if the job isn’t one the technology is suited for, that ends up being the developers fault and they’re the ones who work overtime to put that square peg in the round hole.

It’s scarcely surprising that a) almost nothing works properly, b) there’s constant chaos and that therefore c) almost no-one finds their work rewarding in ways other than money.

I freely admit that a lot of my time, I feel like some sort of thief. I sit in projects which are doomed. They’re more doomed than a plane that’s lost both wings and is on fire. I see the people around the project running about setting more of it on fire as it hurtles towards the ocean and removing more control surfaces and making it worse. And I watch developers switch in and out of seats as we trail wreckage and smoke down. And it’s always been like this. The project is a free-fall disaster before I join and after I leave and there’s just nothing, nothing, nothing I can do to rescue it at that point in time. And yet I’m being paid to be there.

And I feel bad about that. I feel that I ought to be being paid to achieve something. But usually I’m just being paid to sit in a seat in a vehicle performing a ballistic trajectory straight into its crash site. And even if I quit then the next thing will be the same and whoever takes my place in this seat will be in the same situation.

And this is not how I wanted to work. I wanted to build things that people wanted. Not turn up at an aircrash and fill a seat in it for a while. And while it’s financial fulfilling, it’s not very emotionally fulfilling.

Most bridges don’t fall down. Most patients don’t die. Most IT projects are a failure in one way or another. It’s like being a surgeon back in the days before anaesthetics or antibiotics. Most of our patients die… and that’s got to be bad for morale.

And I can’t help but think that this really has its roots in the fact that people think software is simple enough that they can exercise control over it at a level they ought not to be and that they also don’t want to pay what it actually costs. Architects generally don’t compete on price and you don’t get to tell architects that their idea of how strong structural steel is is an underestimate you can feel you can ignore in this project. But people time and time and time again pick the cheapest option for software, design it like they’d design a tin shack and then act surprised when the end results turns out to be flimsy tin shack instead of a tower block.

Originally from http://discuss.joelonsoftware.com/default.asp?joel.3.762361.7, it appears.

Zen and the Art of Software Design

Tue, 27 Nov 2007 08:00:00 +0000

I am currently reading Zen and the Art of Motorcycle Maintenance by Robert Pirsig and it has been chock-full of philosophical insights. One passage in particular jumped out at me due to its simple, but powerful, message:

The material object of observation, the bicycle or rotisserie, can’t be right or wrong. Molecules are molecules. They don’t have any ethical codes to follow except those people give them. The test of the machine is the satisfaction it gives you. There isn’t any other test. If the machine produces tranquility it’s right. If it disturbs you it’s wrong until either the machine or your mind is changed.The test of the machine’s always your own mind. There isn’t any other test.

This quote from the book could relate to almost any field involving technology, but it strikes a particular chord with me in regards to software design and development.

One way this quote can be looked at is an explanation of why a software engineer chooses and, almost more importantly, not chooses a tool for a particular task. We’ve all seen the flame wars that erupt on blogs and message boards about Tool A vs Tool B. Usually each of these pieces of software was developed to fill a specific need for a particular person whose needs were either not fulfilled by existing tools or the existing tools didn’t fit their world view. These original creators were creating “tranquility” for themselves.

When software is created for the reason of creating peace of mind in the developer, the result is often that a tool that doesn’t feel right to them is getting replaced. An example of this is the new web frameworks that have sprouted up over the past few years, mainly Django and Rails. The creators of these pieces of software dreaded the ordeal of writing yet another website or web app in Java or PHP because to them these methods didn’t produce tranquility and just felt wrong for the task at hand. So these developers wrote something that felt “right”.

Since both Rails and Django ultimately were fulfilling the same purpose, other developers that wanted to go down the path of rapid web development had to choose between these two. On the web, whenever there is no clear advantage of one technology over another, conflict arises. You have people entrenched on both sides of the debate of which technology is better. What is not seen is that neither side is inherently right or wrong. Rightness is only determined by what creates tranquility in the person using the tool.

If you like Ruby and the Rails way of doing things, go that way. If you enjoy Python and the Django method, go with that. The main goal is not to choose some arbitrarily correct right tool, because what may be the bee’s knees to one developer may be chopped liver to another. It’s usually not that extreme, but the point is that often arguing over particular technologies is like arguing over favorites colors. The only winners are the ones that choose the tools that suit themselves the best.