Code is Part of the Machine

I intend to share with you a realization that recently crystallized around discussions related to software complexity and stack comparisons.

Computers are very complex lately. They have multiple processors with multiple cores each, communicating with each other to let you wait for idle seconds while that spreadsheet software opens. Or while the kitty “gifs” (likely rebranded mp4’s) load.

Processors can also have microcode, and on top of them sits an operating system, on top of which is our stack, on top of which is our code.

Computers, with all their hardware and software, are machines that imitate other machines – often better than those that inspired them. For instance, almost any text editor in use today we can think of as an impossibly powerful typewriter.

The code we write, and the code we build upon, are part of the machine we are putting together. Yet this is one of the aspects that need to be taken into account when building stuff for the long run: the amount of code present in your production machines is part of the machine.

Complex machines are harder to reason about, keep running, and upgrade than simpler ones.

There are other things to consider, of course, but with the simple comparisons out there of how different programming languages can be more or less complex I think we need to separate things and think about each aspect individually.

Even though this

print("Hello, World")

is easier to read this

#include <stdlib.h>

int main(int argc, char* argv[]){
    printf("Hello, World");
    return 0;
}

All else being equal, the latter is by far the simpler machine.

I’m not certain whether I’ll get to develop this idea as much as I want to in the future, or write about other aspects that need to be considered when you’re building long term systems, so here are some related thoughts:

  • All the code in the production computer, no matter where it comes from, is part of the machine you’re building.
  • All production computers that interact in order to make the system you’re building work are part of the same machine.
  • Yes, this includes the client’s computer, and the client’s phone, with all the apps that are running in the device.
  • Routers and switches are computers.
  • Now that I reflect upon it, I think this is something that Microsoft has known for ages, and part of the reason stuff running on MS systems keep working for relative aeons through upgrades. Some of Raymond Chen’s blog, The Old New Thing has convinced me of that.
  • Ease of writing code doesn’t mean ease of maintaining code doesn’t imply ease of running the code.
  • The most valuable thing to come out of DevOps is the very sane notion that the development team should be involved in making sure the system keeps chugging along.
  • Considering the way early digital computers worked and were programmed, thinking about modern computers as “machines that rewire themselves continuously on the fly” becomes very attractive.
  • The total number of possible machines your production computer can become is part of the complexity of your setup, and is a corollary of the idea that code is part of the machine.
  • Whether your dependencies will become updated or not, the interfaces you depend on die or not, or there are vulnerabilities in your code are a different issue.

If we’re going to reason about all this, it’s important to not muddle our thought process and address as much or as little of the environment of building and running applications as we really mean to.

There is more where that came from, but I don’t know that I’ll have the tools to write it properly. So there we go, I hope this tickles your mind and gives you a new way to think about software systems.

Individuals and Team Performance

Recently, Tim Ottinger posed a question on Twitter regarding the change
in teams and the impact of the change in their effectiveness.

This made me reminisce about some things. I’ve rewritten this post a few times, unsure of how much detail to go into, or whether to publish this at all, but I think at least a few remarks bear mentioning.

I have seen people who accelerate every team they join, and people who put the breaks on every team they join (yes, there is an in-between, but the extremes prove a point). I’ve seen teams swapped whole and the results become “permanently” faster, and I’ve also seen them become slower.

I think the direction and magnitude of change comes down to a few different factors: how the individuals interact with the craft of software development, how they interact with other team members, and how the organization interacts with the software projects they own.

People who respect software development as a craft, would feel embarrassed about making avoidable mistakes and like efficiency in general tend to accelerate software development when they’re included in a project.

People who make their teammates feel comfortable, are willing to learn, teach, and grow together, and don’t feel ashamed of having to learn new things tend to accelerate software development when they’re included in a project.

Organizations with sane rules, specially about developing internally transferable skill sets in collaborators, knowledge sharing, and architectural uniformity tend to suffer less with team changes.

The opposites are opposites, and there’s a lot of nuance that doesn’t fit in this margin, and I can’t claim to comprehend everything completely.

Managers tend to have an big impact in their teams, which should be obvious, but feels worth mentioning. A supervisor who doesn’t know their team, doesn’t teach, motivate, cultivate, and prune their team, makes the company (and the projects the team is responsible for) more fragile.

Teams are bound to change. Requirements change, people move on, people move in. Having every change in the team be a possible cause to a “permanent” slow down in development is a mind boggling, yet alarmingly commonly accepted proposition.

Software development should happen in a way that a change in personnel is more like going trough a bumpy section of the road and less like slashing a vehicle’s tires.

I have sometimes felt that some companies try to treat their employees as exchangeable. This is a liability. Everyone is different, and software should be written in such a way that unique people can bring their unique strengths to the table. This implies, as in any kind of team, that taking care of the composition is crucial. And this goes not only for skill level and skill type, but for attitudes and tendencies. Compare with team composition in your favorite team sport.

*Permanent is in quotes because … well, obviously I’m not looking into eternity. I mean over a period of time when the team has settled into their “day to day performance level” plateau.

Mid year stream of thought

It has been a while since I have written anything longer than two tweets. I notice that I have over ten drafts I’ve not looked at in months, which probably means that they’ll never be published.

I’ve distanced myself from the local developer scene in the Dominican Republic. I feel ambivalent about this, but as it’s been about keeping my head down and working, and I’ve been doing just that, I feel content. I look forward to taming my most demanding endeavors in the near future and reacquainting myself with my local peers.

On the other hand, I’ve been interacting more with developers from overseas. It has been fun, being more exposed to developers from a different culture and using different technologies. Usually my work would involve people from either different culture or different technology, not the combo I’m enjoying right now. More data points to learn from.

My dear friend Lois has been posting poems on her site since around August 2017; I think some from 2018 were lost, sadly. I look forward to the 2nd anniversary of getting new poems every Friday.

Now, the hum of the servers quiets down, and I have nothing else to wait for. Back into the fray.

Before stopping I want to record I felt happy with Nintendo’s E3 announcements; that it’s been crazy juggling around a dozen projects (I don’t want to stop and count, really), but it’s been very fulfilling and fun to push the boundaries. Oh, and that it’s very important to have controlled temperature for work. No HVAC means lost efficiency, lost happiness, wasted time, and a general discomfort about existence when you have to work intensely.

A Desperate Effort (follow up)

Sustaining an increased pace of work (related to your usual baseline) over long stretches of time is grueling.

If it is so different from the usual that it’s a desperate effort – well, I’ve not been able to sustain it as well as I’d’ve hoped. I have managed to tweak my work style to better fit my desire for more productivity though:

  • I have lightweight tasks – pleasant to start and carry out – peppered throughout the week which still help me move forward
  • I read stuff written by smart people, which motivates me to work
  • I read stuff written by people with a lot more money than I have, which motivates me to work smart
  • I read stuff written by people with a lot more output than I have, which motivates me to work hard
  • I have a sharper line between work and play

This has allowed me to move in the general direction I want to go. Less waste, more productivity, more happiness.

 

A Desperate Effort

My twitter bio states: “[I] live where you vacation, and work the way you play.”

Now, I live in a beautiful tropical country and have no plans to change that anytime soon. But I have been thinking about the latter clause.

I already play play. Hanging out, boardgames, books, videogames… nibble by nibble, a wide spectrum of entertainment and feel-good activities.

Why should work also be play?

Well, we must make a living somehow; we might as well make it fun. That’s the logic behind the idea – and I happen to have a line of work I find entertaining. It’s also lucrative – so it stands the whims of starting and stopping… of not taking things completely seriously. I goof off and coast through it.

It’s living the dream. And I felt proud of it. Hence the twitter bio.

And yet, upon reflection, something felt off. The path to my present train of thought was circuitous. Two crucial stops involved a high workload I took on, and remembering this article about extraordinary effort by Eliezer Yudkowsky.

Let’s take a walk.

Working provides sustenance. Leaving wealth on the table is an option, of course; the decision to work less than you can is tied to the hierarchy of values in life. Like most other decisions.

So – what does it say about my values that I work as play? Well, it says I feel very safe about the future; one of the uses for excess wealth Today is an insurance policy for Tomorrow. It says I don’t have much ambition, or any big plans – nothing that would require a solid chunk of cash to make happen.

Well, crap. That isn’t me. Or – it isn’t who I want to be. A clearer explanation:

See, I see the future as a potentially bleak place. I want insurance against a wide range of conditions that can go wrong… and seem to be going wrong.

I see the present as a bleak place. I agree with jaibot‘s name for his blog: Almost No One Is Evil. Almost Everything Is Broken. I’d emphasize: Broken Badly. Lots of things are going wrong right now.

If for some reason you’re not convinced of that: check out WHO’s 2017 world hunger report, reporting 815 million of people living in hunger; of those, there are over 200 million children; 45% of deaths for children under 5 are related to malnutrition; people who suffer malnutrition as children carry consequences for life, both physically and mentally which impact the economic potential – thus making the next generation vulnerable to the same situation (source: Food and Nutrition Bulletin Vol. 20, no. 4, from the United Nations University).

So. Yeah. Next point would be what difference could I make. The answer is: it depends on what I do. I’m all for a healthy placement of the locus of control. A back-of-the-envelope calculation tells me I could easily save a few kids from early life malnutrition even if my ideas about food production, water purification and affordable housing don’t pan out.

Except I work the way you play – I relax, I have a comfortable workload most of the time, and work mostly when I feel like it. I am acting as if I didn’t have grand ideas and ambitions, and that is embarrassing. So much so that I hesitate to write this down.

So I decided to flip myself upside down. What I want to do is very scary:

“First say to yourself what you would be; and then do what you have to do.”
– Epictetus

So we circle back to Yudkowsky’s post about extraordinary effort. Extraordinary effort implies doing things as if your life depends on it.

I’m not sure that anything less than an absurd effort will be enough to change from my current path into the path I want to be in. Part of it is the fact that this realization on how my current actions differ from what I think would be most virtuous didn’t cause the emotional train wreck I associate with successful conversion into a cause.

What’s the plan, then? Well, in my working hours I will not stop working. With the exception of immediate threat of death. Thirsty? Wait. Tired? Poor thing.

I intellectually recognize that no effort is enough. I will leave it circumscribed to my working hours so I will be reasonably sure it won’t affect my health.

So when I work, I will work. My hope is to work so hard I get sick of it by the end of the day, every single day. I’ll try and do this for three months, then check the results. It should be measurable in cash.

The start won’t be gradual. The experiment is timeboxed – I will do this during work hours (8 hours a day), and during work days (5 days a week, except for some already planned outings). I will check my results on June 24th.

If you read the Yudkowsky article, you’ll notice this effort is closer to the “desperate” side of the scale, rather than the “extraordinary” one. I think I am setting up the base needed for the extraordinary work.

I can already feel the weight of this commitment. And I’m guessing the emotional train wreck will happen in the middle of the marathon. I will try to apply stoic principles to avoid suffering – even when in painful situations. We’ll see how things go.

Brief Reflection on the New Year

The new year, as Neil deGrasse Tyson pointed out in ’15, is cosmically insignificant. Which means there’s nothing inherently special about December 31st, or January 1st.

I’d reached that conclusion myself perhaps a dozen years ago, give or take two; so many festivities and landmarks in the calendar make little to no sense when thought about from a greater standpoint.

The magic of many of these landmarks vanished for me.

From a few years ago, my wife’s asked and made me think about goals for the year. Which has been disconcerting for me – I didn’t do that, and the year turning is not really “a thing”. I wound up having rolling kind of goals, more than year-long goals. “By March I will achieve X”, said me, circa November – and other similarly non-calendar-fitting goals. This was a good idea.

“There are neither beginnings or endings to the turning of the Wheel of Time. But it was a beginning”.

– Robert Jordan, The Wheel of Time

I got to choose my beginnings and my endings, which is liberating, realistic, and consistent with reality – people choose things to do all the time.

But if I could write a letter to dozen-years-ago-give-or-take-two-me, I’d say this:

Much like gravity isn’t very important for microscopic beings, cosmic significance is not all that important for people. We’re not working at that scale. We’re working at a socio-cultural scale, a strong context of a few decades with an ever-weaker context bounded in a few millennia.

In that scale there are many important landmarks, and in that scale you can still make your own landmarks, and those still make sense. They work not even because of historical significance – they work because they’re shared context. The landmarks are a ritual by a group or individual, and have meaning bestowed upon them by the minds realizing them.

The origin is not the most important thing, which is what I used to think. The future of the landmark – how long will it be celebrated? – is not the most important thing.

What do we give up, and what do we gain? What are we setting ourselves up for? That’s the strength of the landmark. That’s its use. Now go, younger-me, and think about this for a time in that sleepless New Year’s morrow, while everyone around you is in oblivion and realizations of how different you are from people around you are gnawing.

The landmarks, they can make you belong if you want. They can set you apart if you want.

Happy new year.

Retrospective: Live Code Streaming 2

Today we built up on the previous progress (see what it was here)

I had my twitter app credentials stored somewhere they’d not be streamed, and developed a bit of code to read and use them without showing them on screen. It’s important that those data don’t make it into a code repo! Afterwards, I used the tweepy API to create a stream following a “track”, which is just what we need to get all publications of a particular hashtag.

What remains is to really flesh out the twitter client, make it store the relevant data (see what data is really available), and get what I need.

It seems it won’t be too long – and it’s been longer than needed because I’ve been explaining things.

If you’re reading this, I want you to know that the process has been with trial and error; we found a bug in tweepy’s documentation (it’s fixed in master, which is not published yet), for instance, and I had to work out how to do things properly.

I invite you to join me on the next live stream, I will write on my twitter (@iajrz) when the date is defined.

The lives we project

When using any form of communication we choose what to highlight or not about ourselves.

Social media is heavily skewed towards people having lives full of peaks, be it really good (mostly) or really bad times that can be empathized with.

This is not new – this also happens whenever there’s any kind of meeting happening aftera long time without seeing certain groups of people… be it people you studied with, or distant relatives.

Unlike before, we’re positively bombarded with that kind of phenomena now – the price we pay for being ever more connected, ever better able to communicate with little regard for space and time, and with very little cost. This is a superstimulus.

This is likely to have amplified emotional impacts – if you’re happy with yourself, you’re bound to be happy for other’s successes. If you’re insecure, then not only will you have to see that one person whose life you thought was certainly not going to be more amazing than yours at that rare meeting… you’re going to be hit time and again with that kind of slap, and it’s not hard to see how we’d be left reeling.

Often the young can be most insecure, but also, at this point, the young are heavy users of social media… and while projecting better or more interesting lifes than they have (by choosing to highlight the good times), they can trigger an attempt to compensate from other people, creating a vicious cycle.

Perhaps turning off our social feeds and focusing on chats would be better? Or parhaps we should narrow down our social circles. In any case, a good way to break this is to stop skewing the way you present your life – in which case stopping to do it may be the least painful way to do it. Or trying to feel better about ourselves would make it more joyous to see someone else be absolutely killing it out there.

In any case, thinking about this and taking a grip on the situation may help you feel better and have improve your days, raising your quality of life.

Projecting Conciliation

In some cases getting someone to think in a way similar to yours or changing your mind is especially important. Besides the usual value for your correct reasoning, there’s an immediate decision at hand which is directly dependent on the result of a conversation or argument.

Being non confrontational is really useful in these situations, but requires particular sharpness and preparedness. You need to understand the issue as well as possible, and a quality I can’t quite defined but have observed recently… it’s a mixture of believing in the best cognitive intentions of your counterpart, as well as trusting their smarts and being genuinely curious. There may be other factors that further help you get into the right frame of mind.

The behavior I’ve observed in people who work like this – I’m not particularly good at this approach, although I’ve somehow pulled it off on some rare occasions – is an apparently open but directed curiosity which drives everyone into the meat of the business and enlightens everyone… and is equally likely to change anyone’s mind, except that the driver, the one projecting conciliation, has a deep understanding of the issue at hand that makes them especially likely to be closer to a good answer to the problem at hand.

This kind of attitude, which is also very calm in the conversation, is the best I’ve seen to steer important meetings in the right direction – whatever it might be. It costs a lot of effort, but practice makes master… perhaps doing it in non-crucial subjects is a good idea from time to time.

Retrospective: Live Code Streaming 1

Yesterday I streamed for some minutes while starting the process for building an application which consumes the twitter API.

I’ve not used the tiwtter API before, and I’ve not consumed any APIs from Python… the point is to demonstrate how to learn something new on the run, and how straightforward or not things may turn out.

Up to now, I’ve found out that app.twitter.com is where you register the application, and I chose tweepy as an SDK.

I read on some of the documentation for the proper application behavior, such as the backoff protocol and their unique HTTP 420 error.

Reading on-screen is awkward, I’ve tried to make it lively by commenting on the content and pointing out what calls my attention. As it turns out that reading documentation and exploring is a part of development, I don’t want to hide it from the viewers. There are already a ton of streamers who use their known tools to work on straightforward problems… I hope to enhance the experience bringing to the table something I’ve not yet seen.

As I announced some time ago, this stream is scheduled for Wednesdays at 8:00PM, UTC-4. Next time I’ll have the needed parameters to authenticate to the twitter API and will go on from there, without showing the actual secret values.

If you’re not very experienced and have Spanish as your mother language, I recommend you join the stream and the chat, where we can discuss the information and implementation.