Di’s Lessons in Software Engineering — Abstraction+

10 min readAug 25, 2019

Lesson 1.5. Abstraction+

Programming Languages — Programming Concepts — The Danger of Perfection — Starting As a New Programmer

In the last lesson, I went through why abstraction is one of the most important skills a programmer should learn to master. Now, let’s look at some of the most well-known abstractions in the programming world.

Programming Languages as Abstraction

Yes, programming languages are abstractions. Just picture how computers work at the lowest level. At the level closest to the physical hardware, a program is literally a bunch of 0s and 1s, as represented by state of physical switches, i.e. flip flops. Back when the first computers are conceived and created, this is the level of abstraction people work with to write computer programs: in a representation that can be understood perfectly by the machine but not so much by humans.

Then assembly language came alone, which allows programs to be described in a way similar to how the machine execute the codes. Instead of working with 0s and 1s, assembly language can works with memory location, thus giving programmers power to perform operations such as reading from and writing to a memory location, jumping to a different instruction (which is also stored in memory), and creating arbitrary symbolic names for memory locations.

Later C was born; it removes the need to work directly with memory, and reduces the all-powerful jumps into the loop and conditional statement everyone has now all become familiarized with.

So each of these programming languages abstract away certain low level details from the programmer, and let them focus on what the important task: writing programs. Note that low level languages still exist, as high level languages usually get compiled down to some low level languages. In the end, machine only understands 0s and 1s.

C, in particular, is phenomenal; it’s a very simple language compared to the power it grants to programmer. It’s a perfect example for simple abstraction. However, C does have its problems; many programming languages came along later to fix them either by adding more abstractions on top of C, which tends to make things more complex, or by switching to an entirely different approach, for example, in functional paradigm.

The point is, programming languages are toolkits that provide a collection of basic abstractions to work with. Variable declarations, function call, class, for-loop, conditional statement, module, closure are all such abstractions that allows programmers to do certain things. No programming language is good or bad per-se, and what language you should pick for your project mostly depends on 1) if a language’s conceptual model fits the problem being solved, and 2) the abstraction provided by the language is sufficient to deal with the problem. For example, some programs are very sensitive to the computational performance, and C++ is often the first choice for such programs.

Programming Concepts as Abstraction

Abstraction allows programmers to organize code into units that can be tackled separately, i.e. separation of concerns. At different levels, the unit can be a service, a component, a module, a class, or even just a function, as I’ve done in the earlier example.

Yet, it is still a bit vague how we actually organize code into these units, that’s where programming paradigms come in. These are ways to classify programming languages based on their characteristics. I’ll cover three of them here: structural programming, object-oriented programming, and functional programming.

We’ve seen structural programming earlier in the electricity bill example. As the name suggests, structural programming utilizes control flows, code blocks, and functions extensively to give code into structural clarity. This type of programming is so fundamental, that even with other programming paradigms, you still need to perform the same basic techniques in structural programming.

Then we have object-oriented programming, where the primary form for code organization is a class, instead of functions and control flows. Let’s look at an example first:

// Lang: C++
const Document doc = new PdfReader().Read(filepath);
const AmountDueReader reader = new AmountDueReader(doc);// Result is optional, if the document contains no amount due.
std::optional<float> amout_due = reader.GetAmountDue();
if (amout_due.has_value()) {
  DoSomethingWith(amout_due.value());
}

A little explanation: PdfReader and AmountDueReader are classes, which can be constructed into objects by calling on the class’s constructor. A class’s constructor usually have the same name as the class, so in the above program, the PdfReader and AmountDueReader called on the first two lines are actually the constructor of the class.

The simple way to describe a class would simply be a collection of data and functions. Usually these data and functions can either be public to everyone or private to the class. In the above example, PdfReader has a public method called Read, and AmountDueReader has a public method called GetAmountDue. These public methods, a.k.a. the class’ interface are the only things users really care about the class, even though the class itself would need to have more extensive private code to implement such interface.

Instead of hiding implementation details into functions and abstract an operation into a function, object-oriented programming hides the details into a class implementation and abstract operations into class interfaces.

Finally, there is functional programming, where, as you can guess, most things (if not everything) is a function. The control flow we saw earlier in structural programming is gone, and replaced with functions such as map, filter, and forEach. Most of the variables are also gone now, as they are passed between functions as the return values on one end and arguments on the other.

// Lang: JavaScript
let doc = getDocument()
let pages = doc.getPages(doc)pages.map(page => getContent(page))
  .filter(content => hasAmountDue(content))
  .map(content => getAmountDue(content))
  .forEach(amount => doSomethingWith(amount))

Similarly to functions in math, functional programming focus on the explicit transformation of one set of values to another. This eliminates some major sources of programming bugs, namely mutable states and side effects.

All the three programming paradigms provide a set of basic abstractions to build your program on. Each of them try to tackle a specific type of issue, and just as with programming languages, each works best in a particular problem space.

Before I conclude this section, there is still another group of abstractions that are critical for programmers, and that would be data structures and algorithms. These are the most abstract of all abstractions: e.g. stacks, queues, trees, and graphs. They are just like circles and rectangles: there is no such thing as a real circle in the physical world, but it can be used to represent so many things. Similarly, data structures can be used to understand a lot of more complex problems. For example:

computer memory, as well as the execution of codes, is usually represented as a stack
source code of a program should ideally be a directed acyclic graph
travel route, e.g. highway, railway, and air traffic, and social network, are typical graphs

The same applies to algorithms, which deals with certain computation problems in the most generic forms. Traveling salesmen might be the most well-known of such problem, and problems such as sorting, searching, constraint-based optimization also have wide-range application in real life.

Practical Matters: Imperfect Abstraction

Coming up with good abstraction is an iterative process. In other words, a programmer might not get to the ‘correct’ abstraction for a problem right away (or ever?), but she will keep getting closer to the truth as her understanding of the problem improves over time. The improvement might come from continuous tinkering; it might come from understanding gained by working on a related problem. Usually we don’t understand a problem well right away, and some knowledge can only be revealed by time.

Let’s look at an example. Recall that earlier in the code to extract amount due from utility bill, the content of each page is read, and then a check is performed to see if the page’s content contains the amount due.

for page in pages:
    content = get_content(page)
    if has_amount_due(content):
        amount_due = get_amount_due(content)
        do_something_with(amount_due)

Now, suppose the PDF file my utility company sends me happens to have many blank pages, in which case the content variable will be empty. In that case, I want to skip to the next page if content is empty; this might be because the has_amount_due function assumes the content to not be empty, or does some heavy work that I wish to avoid if possible. As a result, I might change the code to something like this:

for page in pages:
    content = get_content(page)
    if len(content) == 0:
        continue
    # try to get amount_due

Now, suppose there are some other pages in the documents that are essentially blank but not empty; for example, these pages might contain only a page number, a few spaces, or a line saying ‘This page is intentionally left blank.’. It’s no longer sufficient to simply check if the content is empty, instead, we want something like the following to check if a page is essentially blank. Note how the check if performed on the page instead of on the content.

for page in pages:
    if is_blank_page(page):
       continue
    content = get_content(page)
    # try to get amount_due

After a while, we might discover that we might want to skip a page even if it’s not blank, for example, if I know some of the pages in the PDF contain legal or account information that’s not related to what I am looking for. Then it becomes clear the check for is_blank_page should really be for should_skip_page.

for page in pages:
    if should_skip_page(page):
        continue
    # try to get amount_due

The important observation there is this: we don’t and don’t need to arrive at the perfect abstraction right away, even though we should try to do the best we can at the time. So, where should we draw the line? When is an abstraction good enough?

Recall that abstraction is a tool to help achieve other larger goals: creating software that works and is understandable. In that sense, any abstraction is useful if it helps the programmer solve the problem at hand, and the abstraction used also informs the readers (recall code is more often read than written) what is the problem being solved, e.g. check if a page is blank vs. check if the page should be skipped.

If the abstraction is imperfect in hindsight only because the programmer did not grasp some hidden aspects of the problem at the time, it’s totally ok (think of the examples we just give). Trying to make your code adaptive to some future scenarios yet is usually a dangerous move, as it might confuse the reader, and as the software might evolve in a direction differently than you predicted. Still, even though imperfect abstraction from limit of knowledge is a normal thing, imperfect abstraction from a lack of effort is much less tolerable.

The advice in this section seems to contradict with the goal that abstraction should be stable (while implementations keep changing). Here we acknowledge the fact that software evolves and needs to keep evolving (it is ‘soft’ after all!). Very few abstractions would be truly stable, and these might be things like stacks, trees, or graphs. Just like circles and squares, these abstractions are crucial and helpful in many senses, but they usually wouldn’t be sufficient for solving your problem. Your problem will need to have its own abstraction or conceptual model, and this model will likely change every time your problem changes. In this context, the abstraction of your problem should be as stable as possible, so you can have a (relatively) stable model of your problem to work with throughout time.

Starting As a New Programmer

Many new programmers and programmers-to-be encounter this confusion when they finished their first programming course — They don’t know where to go next. On the one hand, there are endless existing and new technologies to learn; on the other, they are pressured to get a job where companies require a vast different set of skills and knowledge for different positions.

What they should do of course depends on the goal of the programmer, but as Uncle Bob Martin said, “The only way to go fast is to go well”. I think instead of worrying about which next step to take, a new programmer should determine what kind of professional they want to become in the future, and prioritize the skills to acquire by working backwards.

Earlier, I stressed the artisanship of software engineering: good software engineers should take pride in creating code that is 1) easy to understand, and 2) easy to change, i.e. soft. Everything else serves to fulfill these goals, includes the power of abstraction, programming languages, paradigms, frameworks, style guides, as well as computer science fundamentals such as data structures and algorithms.

These are also the kind of things important for job hunting as well. When I interview candidates at Google, whether they can solve the interview problem isn’t the only thing I try to observe. Sometimes candidates give a solution that actually works, but it’s unnecessarily complex and hard to understand; as a result, much of the interview time is spent on walking through the solution. Other times candidates might not know exactly how to solve the problem at first, but they have some general ideas of what steps are required, or how the problem given can be reduced to some smaller problems. This is where strong power of abstraction would be very helpful.

That being said, my recommendation would be this: find something that you already understand very well. In my case, I started learning programming by automating my tax preparation work. Given that I understood the process quite well, I had a good mental model of how it should be mapped into programs, i.e. what the abstractions should be, and can thus focus on actually learning the tools and coding out the implementation. Similarly, many people start by building a popular game, e.g. minesweeper, Tetris, or 1024. The process might turn out to be more challenging than expected, but familiarity with the game will give people good context on where to go.