Retrospective: Live Code Streaming 2

Today we built up on the previous progress (see what it was here)

I had my twitter app credentials stored somewhere they’d not be streamed, and developed a bit of code to read and use them without showing them on screen. It’s important that those data don’t make it into a code repo! Afterwards, I used the tweepy API to create a stream following a “track”, which is just what we need to get all publications of a particular hashtag.

What remains is to really flesh out the twitter client, make it store the relevant data (see what data is really available), and get what I need.

It seems it won’t be too long – and it’s been longer than needed because I’ve been explaining things.

If you’re reading this, I want you to know that the process has been with trial and error; we found a bug in tweepy’s documentation (it’s fixed in master, which is not published yet), for instance, and I had to work out how to do things properly.

I invite you to join me on the next live stream, I will write on my twitter (@iajrz) when the date is defined.

The lives we project

When using any form of communication we choose what to highlight or not about ourselves.

Social media is heavily skewed towards people having lives full of peaks, be it really good (mostly) or really bad times that can be empathized with.

This is not new – this also happens whenever there’s any kind of meeting happening aftera long time without seeing certain groups of people… be it people you studied with, or distant relatives.

Unlike before, we’re positively bombarded with that kind of phenomena now – the price we pay for being ever more connected, ever better able to communicate with little regard for space and time, and with very little cost. This is a superstimulus.

This is likely to have amplified emotional impacts – if you’re happy with yourself, you’re bound to be happy for other’s successes. If you’re insecure, then not only will you have to see that one person whose life you thought was certainly not going to be more amazing than yours at that rare meeting… you’re going to be hit time and again with that kind of slap, and it’s not hard to see how we’d be left reeling.

Often the young can be most insecure, but also, at this point, the young are heavy users of social media… and while projecting better or more interesting lifes than they have (by choosing to highlight the good times), they can trigger an attempt to compensate from other people, creating a vicious cycle.

Perhaps turning off our social feeds and focusing on chats would be better? Or parhaps we should narrow down our social circles. In any case, a good way to break this is to stop skewing the way you present your life – in which case stopping to do it may be the least painful way to do it. Or trying to feel better about ourselves would make it more joyous to see someone else be absolutely killing it out there.

In any case, thinking about this and taking a grip on the situation may help you feel better and have improve your days, raising your quality of life.

Projecting Conciliation

In some cases getting someone to think in a way similar to yours or changing your mind is especially important. Besides the usual value for your correct reasoning, there’s an immediate decision at hand which is directly dependent on the result of a conversation or argument.

Being non confrontational is really useful in these situations, but requires particular sharpness and preparedness. You need to understand the issue as well as possible, and a quality I can’t quite defined but have observed recently… it’s a mixture of believing in the best cognitive intentions of your counterpart, as well as trusting their smarts and being genuinely curious. There may be other factors that further help you get into the right frame of mind.

The behavior I’ve observed in people who work like this – I’m not particularly good at this approach, although I’ve somehow pulled it off on some rare occasions – is an apparently open but directed curiosity which drives everyone into the meat of the business and enlightens everyone… and is equally likely to change anyone’s mind, except that the driver, the one projecting conciliation, has a deep understanding of the issue at hand that makes them especially likely to be closer to a good answer to the problem at hand.

This kind of attitude, which is also very calm in the conversation, is the best I’ve seen to steer important meetings in the right direction – whatever it might be. It costs a lot of effort, but practice makes master… perhaps doing it in non-crucial subjects is a good idea from time to time.

Retrospective: Live Code Streaming 1

Yesterday I streamed for some minutes while starting the process for building an application which consumes the twitter API.

I’ve not used the tiwtter API before, and I’ve not consumed any APIs from Python… the point is to demonstrate how to learn something new on the run, and how straightforward or not things may turn out.

Up to now, I’ve found out that app.twitter.com is where you register the application, and I chose tweepy as an SDK.

I read on some of the documentation for the proper application behavior, such as the backoff protocol and their unique HTTP 420 error.

Reading on-screen is awkward, I’ve tried to make it lively by commenting on the content and pointing out what calls my attention. As it turns out that reading documentation and exploring is a part of development, I don’t want to hide it from the viewers. There are already a ton of streamers who use their known tools to work on straightforward problems… I hope to enhance the experience bringing to the table something I’ve not yet seen.

As I announced some time ago, this stream is scheduled for Wednesdays at 8:00PM, UTC-4. Next time I’ll have the needed parameters to authenticate to the twitter API and will go on from there, without showing the actual secret values.

If you’re not very experienced and have Spanish as your mother language, I recommend you join the stream and the chat, where we can discuss the information and implementation.

Learn Ruby The Hard Way

I’m working through the book as a break from other work, and it’s pretty good. It’s aimed to someone getting initiated in programming, which makes it rather easy for me to breeze through the excercises.

What I like most is that the book, just like the rest of the “Learn Code the Hard Way” series, is organized around excercises. You type code, then poke it to break it, fix it and improve upon it.

A good read, and by proxy so is Learn Python the Hard Way, on which this book is based.

Schedule

I usually hate schedules. I like to do things at the moments I feel like doing them.

This results in not being prompt – which often ends with things being forgotten, creating the need to write down and check lists of things to do, which is not a happy overhead. Some times, this results in things not getting done in the optimal timeframe.

Because I need to get some things done – specially my latest goal of bringing little joys to lots of people – I will now adopt a schedule.

I announced on Facebook some weeks ago I’ll be live coding a twitter bot in python. That’s going to be Wednesdays at 20:00 UTC-4. The rest of the schedule is not publicly available – or of public relevance, for that matter. So I’ll keep it to myself.

I’ll talk about the effects it actually has – but I predict I’ll be able to get side projects done at better than present rate.

Dream

I had a dream which left me shaken. I share it here with all of you.

I went into a huge facility, with immense amounts of crafting materials available for use. The materials were in supermarket-style showcases, rows upon rows of them. All kinds of crafting materials could be found: colored pencils, markers, watercolor, pencils of different hardness, oil paint, chalk sticks, all kinds of paper and cardboard, and a myriad of tools to work with all of them.

There were kids running around, with little supervision, getting supplies and going about their business crafting, drawing, painting, designing. Some kids were working in architect tables, frowning with concentration.

Everything was brightly colored and happy. It was a center of study for people who didn’t fit in regular school or didn’t want to be there.

I wept, sobbing about the chance I’d missed by being born too early, about what might have been if I’d had a place to thrive like this instead of becoming atrophied at school. My wife appeared beside me at that point, in pure dream-fashion, and reassured me that not all was lost. In fact, it turned out that we were there because she was invited – something to do with her starting a business and getting first class mentorship or something of the sort.

I calmed down, and we kept walking. I got separated from my wife, as she went to a meeting and I wanted to see behind the next corner. She said she’d be waiting for me. As I turned the corner, and again in true dream fashion, I was suddenly naked and sitting with a backpack in my lap… just when some very important people – who I intuitively knew were very smart – turned the corner discussing something apparently important. I first panicked, then reasoned that this were likely very tolerant people, and plain told them “It seems I got naked by accident”. They looked at me matter-of-factly, and one of them said “That kind of accident happens around here from time to time”. I said, “If you don’t mind, I’ll get dressed now”, and did with clothes that were in my backpack. They weren’t fazed.

I went out of that corridor and noticed a big, burly man threatening a smaller one. He had a gun in his hand, and was pressing it down the smaller man’s trapezium. I rebelled against this abuse – this was no place for that kind of behavior, this was a house for intellect. It’s not nice to be pointing guns at people unless you don’t a better solution. I suddenly “remembered” that the facility owners kept guns everywhere, and went to a drawer to reach for one. Luckily, it was a revolver, so I’d be able to use it. When I got ahold of it, a siren sounded – I woke up while realizing that I was being tested, that they’d wanted to know what I’d do when confronted with such a situation – whether I’d let abuse go unnoticed.

Somehow, while waking up, I had the notion that they’d wanted to teach me, and had wanted to test my mettle and moral fiber. I hope I’d passed the test, for that is a place where I’d like to be: dozens upon dozens of really smart people, hardly bound by the less useful parts of culturally inherited customs, making their own rules and trying to do the right thing for the world and for themselves, working merrily away to stave off death and decay, to make our time on Earth both pleasanter and longer. This is something to aspire to.

There may be some such places in the world. Surely another one can’t hurt? Maybe this is something I should try to help happen sometime in the future?

Recording Events

While dealing with recordings from a recent event (the second edition of OpenSaturday, some pics here), I had a sudden flashback to when I put some time into recording things that were going on.

These are a few points to take into account, to maximize usable footage:

Don’t move the camera

No matter what you do, avoid moving the camera. Before the event starts, look for a good place. Take into account where people are going to be, and make sure that no one will be in the way. Make it point to the interesting place (ie: the stage) and cover as much of the interesting space as possible. If it’s a talk, coordinate with the speaker – make sure their movement range is covered by the camera.

Use more than one camera

The preference is that they overlap. Because something will be wrong with the first camera, and remember you can’t move it. A second camera pointed at the area of interest from another angle should do the trick. Because someone will walk over the range of view of the first camera, or one of the cameras will fall, or the speaker will walk outside of the foreseen territory.

A third one, pointing at a possibly interesting place, like the audience, may be moved from time to time. When you point at a place, count at least three minutes before touching it again. Not in your head. Use a clock.

Those angles should be good enough to provide material for transitions and capturing reactions or otherwise editing the material.

Record continuously

Edition can shorten a long talk or event, but it can’t make it longer. What’s lost is lost.

Run a test drive

Set the cameras to record, simulate some talking from different places, move around, get the recordings and play them. If it’s all good, then it’s all good.

Get the audio

If possible, use an independent audio recording device in addition to the camera’s.

Have Fun Editing

Now you’ll have a lot of material to choose from, and will be able to get good angles and good sound on most of what is going on in the talk or event. The picture will be steady and clear. This is already a lot better that what is usually managed by amateurs.

Avoid an excess of transitions, and look for a narrative. I like to watch stand up comedy videos and take cues from the cuts to the public and angle changes. If we stick to this, and improve upon it, the communities will all be better off, because sharing talks will be less awkward.

Some Comments On Code Optimization

I often code for fun, building tiny toys to amuse myself. Be it simple board games that play themselves (tic-tac-toe and checkers), or some simulations (I have some Windows 7 gadgets laying around), or… well, you get the idea.

In the simulations, for which I developed a taste thanks to a former coworker, I tend to sit down and consider every little calculation I’m going to be performing. I usually run them in an HTML canvas, which both makes them suitable for use as Windows 7 desktop gadgets and lets me have good fun in a healthy environment: It’s easier to e-mail one html file than an exe, and the tools to profile browser javascript code are good to be familiar with.

In any case, one of the things I learned while doing this is that the way in which you run the loops has a tiny impact on performance. It adds up when you try to calculate an immense amount of vectors applied to an immense ammount of particles in a dynamic array… and as we’re running on top of a VM on top of a browser on top of the OS, every little bit counts. Whenever possible, I write the loops in the form

while(n–>0){
// do stuff
}

Why? Javascript implementations, just like the Java VM and the CPython runtime and x86 assembly have special codes to represent a handful of numbers which usually include 0, -1, and 1 through 5 at least. Not a big deal – unless you’re avoiding cache misses. As it’s a cheap optimization to make (doesn’t hurt readability), whenever it’s logical I use it.

This kind of optimization is nice and clean fun, but not necessarily impactful. The best way to optimize code is:

  1. Write it in a readable way. Forget about optimizations.
  2. Measure the hotspots, that is: measure where time is being spent
  3. Optimize that

Of course, before ever writing a line of code, some thought must be given to what you’re going to write. The fastest code is the one which does the least, at any given level. In the example above, the optimization is at the bytecode or assembly level; when you choose the wrong data structure or the wrong algorithm, you affect every level. This is the value of that “algorithms and data structures” class or book which you didn’t pay too much mind to at school.

I’m going to put forward two examples of choice, one of data structure and one of algorithm.

An Example of Data Structure Choice

If you need to organize a list of existing data, you may choose to use a list – after all, the amount of data to be organized could change in time. But different implementations of list have different characteristics. In Java, for example, you have LinkedList and ArrayList implementations of a List (among others). Any implementation of List can be used as an argument for of those you could invoke Collections.sort(List <T> list). This implementations, of course, mirror what you could write by hand if you wanted to. Indeed, I’d urge you to write an ArrayList implementation.

In any case: Which one would you use?

If you chose the ArrayList, pat yourself in the back. To organize data, you need to constantly compare elements, and the fastest way to do that is with a contiguous, indexed block of memory. The difference in performance can be observed in a desktop organizing random integers.

An Example of Algorithm

I’ll share something very real: An e-mail from GNU grep author Mike Haertel in a FreeBSD mailing list in 2010, in which he dissects the reason GNU grep is faster than BSD grep. Long story short: GNU grep executes fewer operations to achieve the same goal. Read it through, I could hardly improve upon Mr. Haertel’s exposition.

Conclusion

Toy around, and have fun with your micro-optimizations all you like. But be careful with the algorithm and data structure you choose.

Sometimes you’re calling some library or external service only to discard the result if some condition isn’t met. This kind of mistake is especially abundant in inherited codebases which accrue functionality over time.

Most important: if code is being slow, use a profiler. Optimize the hot spots, leave the rest alone. How good can this be? Read the answer here. Have fun :^)