Development Environment 4

This is the fourth installment in a series about the components of typical development environments. Part 1 introduces the components and the Text Editor, Part 2 covers the Interpreter, and Part 3 covers the Compiler. This part covers one of the less understood components of a toolchain, the linker. It’s easily overlooked because of the prevalence of interpreted and languages, and those which run in a Virtual Machine which don’t use it, but it’s an integral part of a compiled language toolkit.

The Linker

If you’ve been following this series, you may remember I mentioned that usually, compiled programs don’t have all the code needed to execute them.

Where’s the rest? In libraries.

Libraries are what results when compiling code in a manner that is not executable, but can be referenced from other code. The names used in the programming language specify what names from compiled libraries should be referenced for execution. The linker provides, perhaps unsurprisingly, a link between the program’s source code and the libraries. It simply connects compiled code into a single instance.

There are two kinds of uses for libraries: static, and dynamic. The static use of libraries is easiest, so I’ll cover it first.

Static Linking

The way it works is by outputting a single executable code file which contains both the program’s executable code and the libraries. Executable code files are often called “binary files”, and I’ll call them so from now on.

If you write a program which displays something in the screen, the binary produced by the compiler references the names of the libraries that contain the machine code to do it. If you produce a static linked binary, the libraries referenced and the program binary are stored in a single executable file. The resulting files are big, but can be moved from computer to computer that share the same platform with ease. As many libraries are not included in the Operating System, custom or third party libraries can be included like this for ease of distribution. If that wasn’t done, then the binary couldn’t access the code it needs to perform its operation.

Dynamic Linking

This approach works, as the name states, dynamically. When the program is compiled, it’s not merged with certain – perhaps any – libraries. This produces a small executable file which depends on the libraries being installed on the system to function.

It’s best used when no third party or custom libraries are needed, but can tie the executable firmly to a particular environment and configuration. One of the ways this can happen is when an Operating System update changes a library, which can change the functionality or introduce new bugs. In particular, some applications are certified with a fixed environment, which can make system updates undesirable to preserve the application’s certification. In other words: Dynamic linking makes for a smaller executable file footprint, but makes it brittle in ways that, sometimes, are not acceptable for the applications.

The only linker I know about is called “ld”, and it’s present in unix and unix-like systems. In windows, I’ve only ever worked with dynamic linking, which means that any particular DLLs need to be installed in the target system or packed as independent files when distributing the software.

[EDIT: There seems to be a widely available gratis Windows linker made by Microsoft, called… Linker.exe; I’ve never used it manually. Homepage]

The Rust programming language’s linker can link statically under Windows, but as far as I can tell the linker is included in the language’s compiler, rustc.

[EDIT: the compiler invokes an existing linker in your Operating System, it’s not included in the compiler.]

This is all for the week. Next time we’ll be looking at static code analyzers.

I want to take this end of the week space to invite you to e-mail or comment any doubts you may have; this articles are, of course, subject to revision.

Thank you for reading.

Development Environment 3

This is the third installment in a series explaining the common components of software development environments. To get the sequence, make sure to check out part 1 and part 2 of the series, in which text editors and interpreters are covered.

Today, we’ll talk about another component in the toolchain.

The Compiler

In a way similar to the interpreter, the compiler needs to know about the language you’re working on.

The compiler reads a file in a particular format and outputs a file in another one; this can be thought of as a translation. Compilers usually translates source code to machine code.

The part of the compiler that generates output from the source code – as opposed to executing it as the interpreter does – means that the compiler needs to understand how to write a file that can be executed in a particular environment.

It is common for compilation in a particular environment to produce a file that can be executed in that same environment, but not necessary. When a compiler produces a file that can be executed in another environment, the activity is called “cross-compiling”, to signal that the file is aimed towards a different environment.

In short: A compiler is a program which knows about the programming language you are using and the environment in which it the resulting program is supposed to be executed.

As is the case with interpreters, there can be more than one compiler for any particular language and targeting any particular environment. Here’s a list of some gratis compilers:

  • gcc; this is a widely used compiler collection. It’s a program which can compile from several programming languages and into several platforms. Some of the languages it works with are: C, C++, Objective-C and FORTRAN. It can produce executable for many target environments as well.
  • clang; this is a compiler aimed at the C family of programming languages, and can work with C, C++, Objective-C and Objective C++; it supports several target environments, notable x86, x64 and ARM processor architectures, which most computers and cellphones use.
  • fpc; this compiler is aimed towards the Pascal language and its variants, including Object Pascal. It supports many environments, so much that the “write one, compile anywhere” philosophy is a core idea for the compiler.
  • javac; this compiler is aimed towards the Java programming language, and targets the Java Virtual Machine environment. It’s notable as part of the category of compilers that don’t have a physical machine as a target, but a virtual machine which can then be written in another language and compiled for any desired target. Thus, writing and compiling once would allow a Java program to run anywhere a Java Virtual Machine was compiled to run beforehand.

There are some caveats I’d like to make at this point:

  1. Execution environments are commonly called platforms, and are usually the combination of a particular processor and a particular operating system.
  2. I stated before that the output is usually an executable file. This is not precise information, merely a simplification. The full explanation is that the compiler usually outputs an object code file. It so happens that object code is the code that can get executed by the computer, but it isn’t necessarily formatted in a way that the operating system executes it directly. When not producing executable object code, the compiler is usually producing referable object code.
  3. I said a compiler usually produces object code; but because of the nature of the compiler, taking from a source language and producing some executable output file, when a program produces “bytecode” (object code for virtual machines) or it produces a source file for an interpreter to use (in which case it can be called a transpiler), it’s still a compiler.

Now, object files don’t usually have all the machine instructions that need to be executed in order for the program to function; why this happens and how this fact fits with the successful production of executable programs will be explored Tomorrow, when we cover the linkers. The answer is related to the non-executable object code files that the compiler can produce.

This is all for Today. I hope this was entertaining and informative.

Development Environment 2

In the previous installment, we saw a list of types of tool used in a development environment, a bit about the purpose of the Text Editor, and a list of existing gratis text editors for several platforms.

After you have the source code, (i.e. text files describing what your software is supposed to do in a particular language), you still can’t execute it. One of the tools that help you make your code executable is the interpreter.

The Interpreter

The interpreter is a tool which understands a particular programming language and can execute the instructions. So, if your source code states that a picture should be shown on-screen, the interpreter shows the picture.

Basically, it’s a program which you ask to run through your file and do what your file states should be done. Many programming languages have interpreters, and sometimes a single programming language can have many interpreters. Some languages can only be worked on with interpreters; those are called, rather unsurprisingly, interpreted languages.

An interpreter usually contains either a lot of functionality, the capacity to use external libraries to provide a lot of functionality, or both.

A short list of gratis interpreters follows:

  • python; As well as being the name of the programming language (Python), the interpreter is called python. This naming is typically used for the old version, i.e. Python 2.x
  • python3.5; that’s the name of the latest (as of today) stable Python interpreter for the 3.x version of the language.
  • IronPython; an interpreter for the python 2.x language which uses the .Net CLR
  • node; it’s an interpreter executable based around the v8 JavaScript engine
  • perl; it’s an interpreter for the perl programming language
  • racket; an interpreter for the racket progamming language
  • lispy; an interpreter for a LISP written in python

As you can see, it’s common to have the interpreter executable and the programming language share a name, and there can be several interpreters for the same programming language. This is because the interpreter provides the facilities for the code to run; different interpreters can provide different facilities for the same code to perform the same function across diverse conditions, like different installed Operating Systems.

Unlike with the text editor, each tool of this step is built to work with a particular programming language. It is so because the tool needs to know about the language to be able to understand the source code, but the text editor needs only to understand text input and how to store plain text files.

Usually, the interpreter has no output. There are two notable situations in which you have output:

  • Generating output is the interpreted program’s intent, which means it’s the program’s output even if it’s generated through the facilities provided by the interpreter
  • Things go wrong; in which case you get error messages

Things can go wrong in many ways, but chiefly:

  • a mistake was made when writing the program’s text, such as a typo
  • a rule of the programming language was broken
  • the nature of the program being run makes it not work properly (for example, having a loop that never ends and never does something productive), in which case it’s not a problem directly related to the interpreter but to the written program

This track of the toolkit (editor -> interpreter) doesn’t have any more dependencies to produce running software. Just the text editor and the interpreter. If an interpreter can run in several platforms (as is often the case), it’s very likely that the same code can run in those same platforms. Other tools can be used on the code and running program, like the debugger or the static code analyzer. Those tools will be covered in a few days.

Tomorrow, we will cover the compiler.

Development Environment 1

Since I started cross-pollinating with people who learned in a different environment from mine, I remember questions and remarks similar to this:

I’m using DevC++, what compiler are you using?

This evidences a lack of knowledge about what makes a development environment… a fragility in the ability to get things done that hangs for dear life on a particular way to do things with particular tools instead of knowing how the pieces fit together and being able to mix and match. Hopefully a bit of context can help fill the gaps.

This is the start of a series of posts aimed towards anyone who wants to start developing and needs to make sense of what they need to start developing software and how the pieces fit together, so that a particular tool or collection of tools isn’t what defines the reader, but what helps them be productive in a particular environment and circumstance.

The tools most commonly used in developing are: a text editor; an interpreter, or a compiler and linker; a debugger; an automatic build tool; a profiler; a static code analysis tool.

This series will not go in depth on how the tools work. Instead, I hope to give an overview of what they do, and some examples of known tools in the category. Almost every category of tool used in software development depends on the output of another one to work; thus the combination of tools that get you from code to finished product is called a “toolchain”.

Some programs bundle many or all of a particular toolchain together; some even let you swap tools within a category to better fit your needs and preferences. Those are called IDEs (Integrated Development Environments), and I’ll cover them last.

The Text Editor

First of all, with almost all certainty, your chosen language’s toolchain starts its with plain text. This means that you can write the code for it in any text editor. Many text editors provide “syntax highlighting”, which means that they understand enough of what certain programming languages mean and highlight the different elements of the source code for ease of reading. Just to be clear, the source code is what you actually write in a particular programming language to specify the steps that the final program needs to execute.

Here is a brief, vastly non-exhaustive list of gratis text editors:

  • TextEdit, the out-of-box MacOSX editor
  • Notepad, the out-of-box Windows editor
  • Gedit / Leafpad / Kate, the out-of-box editors which usually install with GNOME, XFCE and KDE desktop environments in Linux
  • Notepad++, an editor available for windows with some good features aimed at software development
  • Emacs (My editor of choice), a multi-platform and customizable text editor with its own learning curve. I recommend if you’ll use this put some time apart to learn the basics, then start learning to program, then put some time apart to learn to ease your programming using external modules.
  • Vi (I’ve not used this one extensively; it’s most used incarnation is Vim), is also multiplatform, also customizable, also has its own learning curve. It’s style is very distinct from Emacs’. As with Emacs, though, I recommend you set apart time to learn it properly before heading into progamming, then learn to customize it with plugins to ease your programming. It’s particularly important in that it’s standardized in POSIX, so you can expect consistent behavior.

The output of this program is a text file with source code, which is then processed by either the interpreter, the compiler, or the static code analyzer.

This is all for today. I expect we’ll cover the interpreter Tomorrow.

Discipline

Some years ago, in a Japanese-Colombian conference, a guy who is half Japanese and half Colombian mentioned something they apparently say in Japan, roughly translated as:

Sooner or later, discipline overcomes intelligence.

This has resonated with me, and fits with anecdotes from elsewhere, echoing Paul Graham’s take that determination is the number one predictor for startup success, the tales of John Carmack’s incredible dedication and hard work, and the situations in which disciplined work and study have served me well.

I believe that discipline can be cultivated, like many other habits, so I will work on it this year.

I will start by writing a post each weekday, with no restrictions as to time, length and subject. I will then impose a schedule, and a length; I don’t really think a restriction on subject would help. The objective is to build discipline in this area, and grow it into others, then look at the fruits of it and see if it is worthwhile.

Last time I checked, I had above-average IQ; hopefully cultivating discipline will be a multiplicator of that gift, and this will be an better year than otherwise for that.

I’ll archive the texts written with discipline cultivation in mind under the “Daily Writing” category for future reference.

Starting the blog

In order to write more, and be able to share better, I dropped the development of my static site compiler and moved to wordpress.

Over the years I’ve set many rules and goals to achieve before starting to write; among those:

  • Setting up a multi-language website. I’m a native Spanish speaker in a Spanish speaking country, but the broader audience is English speaking.
  • Setting up a static website, and the appropriate generator, to make the maintenance painless — this one was so misguided it’s actually hilarious.
  • Writing a backlog of content, and work with that content in several different ways, including but not limited to: classifying it, running it through trusted people, embellishing it.
  • Getting some OSS projects up and running so I could talk about them — more hilarity ensued by not thinking this one through and noticing a lack of community around my obscure development.
  • Generally trying to improve who I am, so I could project a better picture of myself while remaining truthful. Part of this is related to a severe impostor-syndrome episode, caused in no small part by working on in-house, backoffice software during all of my career.

So what I’ve decided is this: I’ll throw everything overboard and start. I expect a backlog will eventually form, and everything will be nicer as I work and polish my workflow. I’ll write about many things, including what I’m doing now, what I’m reading about, and insights I develop.

This is a start which almost aligns, with a slightly amusing off-by-one error, with the start of 2016. This is a coincidence, as I’d been working on setting up the php and MySQL installation and making sure they played nicely with Lighttpd for a few days, and festivity-related “chores” intervened. I’ll roll with it, though and try and make a nice run.