
- [Introduction](#introduction)
- [What is `make` and *Makefile*](#what-is-`make`-and-*makefile*)
- [Advanced usage](#advanced-usage)
- [Conclusion](#conclusion)
## Introduction

Many developers may recall the first day they started working on a new project. After cloning the main repository, there comes a point when you have to enter a lot of commands with certain flags and in a specific order. In most cases, it’s hard to grasp what is going on without a description of the commands. For example:

```bash
# Bash
touch ~/.bash_history
ufw allow 3035/tcp || echo 'cant configure ufw'
ufw allow http || echo 'cant configure ufw'
docker run \
  -v /root/:/root/ \
  -v /etc:/etc \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/tmp:/var/tmp \
  -v /tmp:/tmp \
  -v $PWD:/app \
  --network host \
  -w /app \
  --env-file .env \
  ansible ansible-playbook ansible/development.yml -i ansible/development --limit=localhost -vv
grep -qxF 'fs.inotify.max_user_watches=524288' /etc/sysctl.conf || echo fs.inotify.max_user_watches=524288 | tee -a /etc/sysctl.conf || echo 'cant set max_user_watches' && sysctl -p
sudo systemctl daemon-reload && sudo systemctl restart docker
```

These commands are just a small portion of the project deployment process. The commands themselves are extensive and contain several flags, as shown in the example above, making them not only difficult to learn but also difficult to enter manually. Constantly maintaining documentation becomes more challenging as the project grows; it inevitably becomes outdated, and the entry barrier for newcomers increases since no one can remember all of the project's details. Some of these commands must be used on a daily basis, if not multiple times per day.

Over time, it became clear that we desperately needed a tool that could maintain such commands, and provide convenient shortcuts and self-documentation of the project. This is precisely what *Makefile* and the `make` utility have turned into. In this guide, I'll show you how to reduce the deployment to a few short and straightforward commands using these tools:

```bash
# Bash
make setup
make start
make test
```

## What is `make` and *Makefile*

*Makefile* is a file that is stored in the repository alongside the code. It is usually placed at the project’s root. It acts both as documentation and as executable code. The Makefile hides the implementation details and manages the commands, and the `make` utility runs them from the Makefile in the current directory.

`make` was originally designed to automate the building process of executable programs and libraries from source code. Most *nix distributions have it by default, which has contributed to its extensive use. Later, it turned out that this tool is convenient to use in other development projects because the process is essentially the same in most cases - automation and building applications.

The `make` has become a standard for many developers, especially for those working on large projects. Examples of makefile can be found in projects such as [Kubernetes](https://github.com/kubernetes/kubernetes/blob/master/build/root/Makefile), [Babel](https://github.com/babel/babel/blob/main/Makefile), [Ansible](https://github.com/ansible/ansible/blob/devel/Makefile), and, of course, everywhere on [Hexlet](https://github.com/Hexlet).

### *Makefile* syntax

`make` runs targets from a *Makefile* that contains the following commands:

```makefile
# Makefile
target1: # target name, you can also use kebab-case or snake_case
	command1 # it's very important to use tabs to indent 
	command2 # the commands will be executed sequentially and only if the previous one was successful
```

However, it's not enough to just start using a Makefile in a project. To make its implementation more efficient, build a target-oriented command structure and give the targets semantically relevant names. At first, moving commands to a Makefile may result in all commands being merged into a single one with a vague name:

```makefile
# Makefile
up: # deploying and launching
	cp -n .env.example .env
	touch database/database.sqlite
	composer install
	npm install
	php artisan key:generate
	php artisan migrate --seed
	heroku local -f Procfile.dev # project launch
```

Several actions take place here at once: creating a file with environment variables, preparing the database, generating keys, installing dependencies, and launching the project. Since this is impossible to understand from the comments and target name, it’s best to separate these  commands into different independent targets:

```makefile
# Makefile
env-prepare: # create .env file for secrets
	cp -n .env.example .env

sqlite-prepare: # prepare a local database
	touch database/database.sqlite

install: # install dependencies
	composer install
	npm install

key: # generate keys
	php artisan key:generate

db-prepare: # upload data to the database
	php artisan migrate --seed

start: # run the app
	heroku local -f Procfile.dev
```

Now that the commands are divided into targets, you can individually install dependencies with the `make install` command or run your app via `make start`. But the remaining targets are only required during the project's initial deployment and must be performed in a specific sequence. In the Makefile world, the target has the following prerequisites:

```makefile
# Makefile
target1: target2 # here established the command dependency, target1 depends on target2
	command2 # target2 will be executed only if target2 command was successful

target2:
	command1
```

Commands will only be executed in the specified order and only if the previous command proves to be successful. Therefore, you can add a `setup` target to combine all the necessary actions:

```makefile
# Makefile
setup: env-prepare sqlite-prepare install key db-prepare # you may refer to the targets described below

env-prepare:
	cp -n .env.example .env

sqlite-prepare:
	touch database/database.sqlite

install:
	composer install
	npm install

key:
	php artisan key:generate

db-prepare:
	php artisan migrate --seed

start:
	heroku local -f Procfile.dev
```

Now it is enough to deploy and launch the project with two commands:

```bash
# Bash
make setup # will run sequentially: env-prepare sqlite-prepare install key db-prepare
make start
```

The project commands and flags are combined into a *Makefile* as a result of the *Makefile's* work. It ensures the correct execution order, regardless of the languages or technologies involved.

## Advanced usage

### Fake target

Using `make` in a project may one day lead to the error `make: <target name> is up to date.`, although everything is written correctly. This is frequently related to the existence of a directory or file that matches the target name. For example:

```makefile
# Makefile
test: # the target in the makefile
	php artisan test
```

```bash
# Bash
$ ls
Makefile
test # the file system contains a directory with the name of the target in the makefile

$ make test # an attempt to run tests
make: `test` is up to date.
```

As stated previously, `make` was designed to build programs from source code. Therefore, it searches for a directory or file with the given name and attempts to create a project from it. To alter this behavior, you need to add a `.PHONY` pointer to the target at the end of the Makefile:

```makefile
# Makefile
test:
	php artisan test

.PHONY: test
```

```bash
# Bash
$ make test
✓ All tests passed!
```

### Running commands consecutively and ignoring errors

You can run commands one at a time: `make setup`, `make start`, `make test` or all at once, space-separated: `make setup start test`. The latter method works as a dependency between commands, although it is not documented in the Makefile. Difficulties may arise if one of the commands produces an error that must be ignored. In the previous examples, such a command was to create an .env-file when deploying the project:

```makefile
# Makefile
env-prepare:
	cp -n .env.example .env # if the file has already been created, using this command again will result in an error
```

The easiest (*but not the only*) way to "cover up” an error is to use a logical OR in the Makefile:

```makefile
# Makefile
env-prepare:
	cp -n .env.example .env || true # any result of command execution is now considered successful
```

Be cautious applying such hacks so that you don't shoot yourself in the foot in more complex scenarios.

### Variables

Configuration parameters, path indicators, and environment variables are often substituted into commands, and `make` enables you to handle this as well. Variables can be written directly in the command within the makefile and passed when called:

```makefile
# Makefile
say:
	echo "Hello, $(HELLO)!"
```

```bash
# Bash
$ make say HELLO=World
echo "Hello, World!"
Hello, World!

$ make say HELLO=Kitty
echo "Hello, Kitty!"
Hello, Kitty!
```

Variables can be optional and have a default value. They are commonly declared at the beginning of the Makefile.

```makefile
# Makefile
HELLO?=World # the question mark indicates that the variable is optional. The value after assignment can be omitted

say:
	echo "Hello, $(HELLO)!"
```

```bash
# Bash
$ make say
echo "Hello, World!"
Hello, World!

$ make say HELLO=Kitty
echo "Hello, Kitty!"
Hello, Kitty!
```

Some variables in the *Makefile* have names other than the system ones. For example, `$PWD` is referred to as `$CURDIR` in the [Makefile](https://github.com/hexlet-basics/hexlet_basics/blob/3f4635bf629e2676efe547c9a01c22a2573eaebd/Makefile#L35-L39):

```makefile
# Makefile
project-env-generate:
	docker run --rm -e RUNNER_PLAYBOOK=ansible/development.yml \
		-v $(CURDIR)/ansible/development:/runner/inventory \ # $(CURDIR) is the same as $PWD in the terminal
		-v $(CURDIR):/runner/project \
		ansible/ansible-runner
```

## Conclusion

In this guide, we covered the main features of *Makefile* and the `make` utility. A deeper understanding of this tool will reveal many of its other useful features, such as conditions, cycles, and importing files. Makefile will be a great help in standardizing generic instructions in companies where multiple projects are written by different teams at different times: `setup start test deploy ...`.

Because the Makefile can describe multi-line commands consecutively, it may be used as a "universal glue" between language managers and other utilities. The widespread use of this tool and its overall simplicity allows you to quickly implement it into your project without making any changes. However, Makefile can be extremely large and complicated, as shown by the following real-world applications:

* [Codebattle](https://github.com/hexlet-codebattle/codebattle/blob/master/Makefile)
* [Babel](https://github.com/babel/babel/blob/main/Makefile)
* [Kubernetes](https://github.com/kubernetes/kubernetes/blob/master/build/root/Makefile)

### Additional materials

* [Modern Make Handbook](https://makefile.site) — a summary of the documentation

Makefile examples from this guide were taken from:

* [Hexlet SICP](https://github.com/Hexlet/hexlet-sicp/blob/master/Makefile)
* [Hexlet Basics](https://github.com/hexlet-basics/hexlet_basics/blob/master/Makefile)


makefile-as-task-runner

A guide to the basics of make and Makefile for use in your own projects

This guide will explain how using Makefile will reduce the process of deploying a project to a few short and clear commands

What Is a Makefile, and How Do I Use It?


Fitting active and regular exercise into your daily schedule may seem difficult, at first your body will resist: aching muscle pain, fatigue, unwillingness to continue. If you go for a run for the first time in decades, the body will clearly tell you: “you don't need to do this”. This is a natural reaction. An evolutionary defense tool against overloads and hazards.

Any physical exercises or sports classes teach people to cope with this reaction and create their workout routine by listening to their body needs.

If you start active and regular mental exercises (for example, learning programming), then, at first, your brain will resist it. The brain has much more resistance tools than the muscles: it can plant thoughts that you're incompetent, that "this is not mine, and it's better for me to do something else," or "I don't have a mathematical mindset" and so on.

This article is about mental resistance to problems and how to overcome them. It consists of four steps:

1. **Self-awareness.** Notice, be aware, become a scientist
2. **Mindset.** Understand what success depends on
3. **Openness.** Exclude one of the main conditions of mental block
4. **Progress**. Accept the realities of the learning process.

- [Step 1: Self-awareness](#step-1:-self-awareness)
- [Step 2: Mindset](#step-2:-mindset)
- [**Step 3: Openness**](#**step-3:-openness**)
- [**Step 4: Progress**](#**step-4:-progress**)
- [References](#references)
## Step 1: Self-awareness

First of all, it’s necessary to sharpen your awareness of emerging mental resistance. Treat your body and mind as a scientific experiment: they give you, the observer, some signals, then they react to actions and environment.

![Мысли и их разнообразие](/assets/images/learning/learning_1_en.png)

Try to notice or even write down the conditions under which such thoughts typically occur. A lot of things affect the mental state, including physical condition, work environment, sleep, nutrition, stress, goals, relationships, etc.

Keeping a journal can be a good idea. Write everything about your everyday life: how much you sleep, what you do, what you eat, how much you read, how you relax, and how is your training going.

It will be great if you can identify the reasons for good and bad days. Sleeping less than 7 hours plus meeting someone before lunch almost always have something with learning problems. Or physical activity and reduced consumption of sweets have something with successful training. Perhaps this is not a mere correlation, but a causality.

Do your own experiments and feel like a scientist.

## Step 2: Mindset

[Carol Dweck](https://en.wikipedia.org/wiki/Carol_Dweck), Stanford University psychologist, identified two core mindsets: fixed mindset and growth mindset.

![Виды мышления](/assets/images/learning/learning_2.png)

Mindset is a collection of beliefs, ideas and concepts held by oneself.

**Fixed mindset** is characterized by such beliefs:

- I am who I am and people don't change
- success relies on innate talent
- people fit into specific categories: techies, humanitarians, strong and weak, smart and stupid, etc.
- I should avoid changes and challenges to prevent the possibility of failure
- efforts are worthless
- I need to choose a path of the least effort
- being criticized is bad
- I have every right to complain about problems and failures

**Growth** **mindset** is characterized by such beliefs:

- I’m able to change, my abilities and intelligence can be developed over time if I put my mind to it
- success can be achieved through effort
- categories of people are more likely approximate descriptions of the current state or areas of interest, they don't define anyone's fate
- I willingly embrace challenges, it’s an opportunity to grow
- effort leads to mastery
- I believe failures are just temporary setbacks and learning opportunities
- constructive feedback as a source of information
- complaints make no sense and distract from the things that really matter

You can notice a recursive trap: fixed mindset makes a person think that his fate is predetermined and he won’t be able to change.

Studies of mindsets lead to an interesting conclusion: the quality of life and satisfaction depend more on a subjective attitude, and *not on talent or intelligence.* In other words, a person can develop a growth mindset with beliefs and rational evidence.

Use your emotional resistance ("probably, this is not my thing", "it's obvious that I'm not a tech guy...", "I would never make it as a programmer") as a starting point in your mindset transformation.

Resistance, problems and failures aren’t going anywhere. Any professional who has achieved something has thousands of times more problems and failures than a beginner who has given up. They are distinguished primarily by their attitude to these problems.

It's fine to feel resistance. This is a sign of growth. Don't miss the opportunity to grow.

## **Step 3: Openness**

Forget the myth that hard work and tons of books you've read will make you an expert. The shortest way to get into learning stagnation is to isolate yourself from other students and teachers.

![Процесс обучения](/assets/images/learning/learning_3.png)

1. **Ask questions.** If you learn programming at Hexlet you will get an answer anyway — our mentors answer all questions after each lesson. Formulating a question also helps to sort everything out and bring you closer to a solution. An abstract problem in the head and the one put into text are different things.
2. **Communicate.** You might call yourself an extrovert or an introvert, but every person needs socializing more or less, and this is crucial in studying. It is very helpful to find other people with similar problems to yours, to know about their progress and experience. Hexlet has a [big friendly chat](https://slack.hexlet.io) for this purpose.
3. **Accept and share your experience.** Find someone who is in a similar situation or under the same conditions. Right now someone like you is trying to learn programming or a new language. Someone is experiencing similar difficulties. Consider starting your own blog.

## **Step 4: Progress**

The process of exploring new areas can sometimes seem too slow or even endless. In the essay ["The Learning Process"](https://rakhim.org/process-of-learning/) I had an interesting analogy describing the process of learning new subjects. I recommend you to read it if you ask yourself the same questions:

- Which book should I start with?
- I kind of understood each topic separately, but I simply don't get how they are connected. And why did I even study it, it makes no sense to me.
- I study a lot but that's not really helping!

![Пазлы](/assets/images/learning/learning_4.png)

I also advise you to read Kirill Mokevnin's article on [effective reading of professional literature.](https://hexlet.io/blog/posts/how-to-read-professional-literature-more-effectively)

A temporary lack of visible progress is OK but a long-lasting one is a sign of serious problems. For example, if after six months of learning programming you still can't write and run a simple program, it's about time to make changes to the learning process. Try as soon as possible (it doesn't have to be perfect!) to create your own project, application, website, etc.

A good strategy is to keep track of progress by recording your goals, progress and successes. Review it from time to time and compare yourself with your past self. If you find past problems a little funny or you are even a bit [embarrassed](https://hello.rakh.im/how_to_evaluate_personal_growth/) — that's awesome! You've obviously grown up. If the old problems haven't gone away, then it's worth changing the approach, conditions and tools of training.

As it often happens, all such topics can be summed up in a short ancient Greek saying: γνῶθι σεαυτόν — **know yourself**.

## References

I recommend you these books if you'd like to learn more about the subject:

1. [Carol Dweck](https://en.wikipedia.org/wiki/Carol_Dweck#References), [Mindset: The New Psychology of Success](https://www.amazon.com/Mindset-Psychology-Carol-S-Dweck/dp/0345472322).
2. [Mihaly Csikszentmihalyi](https://en.wikipedia.org/wiki/Mihaly_Csikszentmihalyi), [Flow: The Psychology of Optimal Experience](https://www.amazon.com/Flow-Psychology-Experience-Perennial-Classics/dp/0061339202/ref=sr_1_1?crid=2SNPIPBAN139T&keywords=flow+the+psychology+of+optimal+experience&qid=1650616340&s=books&sprefix=Flow%3A+The+Psychology+of+Optimal+Experience%2Cstripbooks-intl-ship%2C217&sr=1-1).
3. [Viktor Frankl](https://en.wikipedia.org/wiki/Viktor_Frankl), [Man's Search for Meaning](https://www.amazon.com/Mans-Search-Meaning-Viktor-Frankl/dp/0807014273/ref=sr_1_1?crid=1EYLE5AI3M18L&keywords=Man%27s+Search+for+Meaning&qid=1650616396&s=books&sprefix=man%27s+search+for+meaning%2Cstripbooks-intl-ship%2C210&sr=1-1).


learning

Attitudes and mindsets set the conditions for growth.

Sometimes, your mind makes you doubt yourself when problems arise. That’s OK, you just need to learn to perceive it.

How to Learn to Cope with Negative Thoughts


Deployment is the process of web service “deployment”, e.g. of a website, in the operating environment. An operating environment is a place where a website is started and available for requests. This can be ready-made hosting as well as your own server infrastructure.

*Not only web services but also any services available over the network are deployed. Even if this network is internal and not available for requests over the Internet.*

To understand deployment, you need to understand the code life cycle. The application code is built on the developer's workstation and run in another place called production. Production is a startup environment (also called a live environment). In the case of a simple application, it may consist of a single server, but for truly complex applications it can be thousands and tens of thousands of servers.

How does it work? Developers add the code to the information repository. At some point, they decide it's time to bring it to production. This can be done on a regular schedule such as every two weeks or just as needed up to the release after each change. In many cases, the number of deployments depends on the automation level, i.e. to which extent the process is easy to conduct and rollback. At Hexlet, deployments are performed almost after each change, approximately three deployments a day.

When developers decide now it's time, they create a release. Release usually refers to a Git tag that marks things to be deployed. Namely, changes added to a master branch after the tag has been created won't affect the tag itself, which means that we are sure about what we will be deploying.

{/* image */}

For static websites or a separate frontend (HTML, CSS, and static files only), deployment comes down to updating the code on the server. In the case of backend deployment, the database is connected at the very least. In general, deployment can be a complicated procedure taking decent time. In distributed systems consisting of many independent web services, there is no regular deployment at all - each part of the application is deployed (released) independently.

*It's worth mentioning that such PaaS platforms as Heroku take the deployment completely upon themselves. There you just have to make a commit, and then everything will happen by itself. The price for this is the cost of the platform itself.*

- [Deployment stages](#deployment-stages)
- [Automation](#automation)
- [Zero Downtime Deployment](#zero-downtime-deployment)
- [Links](#links)
## Deployment stages

### Code delivery to the server

There are different options for code delivery to the server, depending on its packaging. Sometimes the code is directly transmitted to the server as a set of files, but more often it's updated via Git. Back in the day, the deployment method through standard package managers of Linux distributions was popular. It's still in use today and best suited for certain situations.

* Git: *git checkout tag-name*
* Docker: *docker pull image-name:tag-name*
* Apt (package): *apt-install application-package-name*

### Database update

In general, a new application version requires changes in the database. To do this, migrations - special scripts containing database update rules - are run during (or before) deployment. For instance, SQL scripts:

```sql
CREATE TABLE car (
    id INT NOT NULL PRIMARY KEY,
    license_plate VARCHAR NOT NULL,
    color VARCHAR NOT NULL
);

ALTER TABLE owner ADD driver_license_id VARCHAR;
```

### Start and stop

Somewhere in this process, the old version stops, and the new one starts. If we first stop the old version, then execute migrations and start the new version, we will get service downtime. This is a really common thing for many, but it can be sensitive for business and frequent deployments. So, the most advanced projects don't stop during deployment. There is a description below of how to do it.

## Automation

Deployment should be automated as much as possible; Time To Market, a key characteristic of business-oriented applications, depends on it. The faster and more often we deliver changes to the user, the better. The faster we check hypotheses, the faster we introduce changes, and the faster the money invested in development pays its way. Developers are afraid to perform deployment without automation, it becomes a burden that leads to a decrease in deployments and regular stress for the whole team, working long hours.

There are three main automation ways:

1. Using utilities created for specific languages. For example, it's Capistrano in Ruby, one of the first and best-known utilities of its kind, which has become popular far beyond Ruby. The main problem with such tools is strong binding with the language
2. Using Ansible with an [already built-in module for deployment](https://docs.ansible.com/ansible/latest/collections/community/general/deploy_helper_module.html). It's relevant for most deployment situations on managed servers
3. Kubernetes-type orchestration systems. If they are used, you can't do without automated deployment

However, even if automation is executed, the task of "running deployment" remains. The start is also automated. There is a whole approach called [Continuous Delivery](https://en.wikipedia.org/wiki/Continuous_delivery). It's difficult to implement and is not suitable everywhere, but if it works, you can forget about deployment. It's performed entirely on its own without human involvement. The main thing in this option is good monitoring and alerting system.

```yaml
# https://docs.ansible.com/ansible/latest/collections/community/general/deploy_helper_module.html#examples
- name: Initialize the deploy root and gather facts
  community.general.deploy_helper:
    path: /path/to/root
- name: Clone the project to the new release folder
  ansible.builtin.git:
    repo: ansible.builtin.git://foosball.example.org/path/to/repo.git
    dest: '{{ deploy_helper.new_release_path }}'
    version: v1.1.1
- name: Add an unfinished file, to allow cleanup on successful finalize
  ansible.builtin.file:
    path: '{{ deploy_helper.new_release_path }}/{{ deploy_helper.unfinished_filename }}'
    state: touch
```

## Zero Downtime Deployment

If no special steps are taken, each deployment will cause the service to stop (possibly partially). At this time users will either see an error or a message that an update is in progress. But this does not happen on most major Internet services. Why? Due to the implementation of the “Zero Downtime Deployment” approach (downtime means service operational downtime).

Zero Downtime Deployment is when the service never stops running, but in the meantime, it's being updated. This is achieved by simultaneously running the old version and the new code. Namely, when an application is deployed, the new version arises next to the old one. And only when the automatics make sure that the new version has started and run, the old version is stopped. The following is required to perform this procedure:

1. Infrastructure. You need a load balancing to switch traffic (incoming connections from browsers or other systems) between the old and the new version of the code. And it's desirable to have at least two servers, although it's not required
2. Deployment. Zero downtime deployment process is much more complicated than the usual one. The easiest way to execute it's on orchestration systems such as Kubernetes
3. Code culture. It's impossible to ensure non-stop operation without a certain culture of code writing. For the old and the new version to work simultaneously, you need to keep an eye on all interfaces. It's important to maintain backward compatibility (working with API, database, queues, and, in general, with any storage)
4. Database. It should be backward compatible between the old and the new versions. All migrations are forward-only (you can't undo them!) and for addition only. You can't delete and update (rename, change the type of) columns and tables

```yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: tomcat-deployment-${TARGET_ROLE}
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: tomcat
        role: ${TARGET_ROLE}
    spec:
      containers:
      - name: tomcat-container
        image: tomcat:${TOMCAT_VERSION}
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /
            port: 8080
```

## Links

* [Ansible Deploy](https://docs.ansible.com/ansible/latest/collections/community/general/deploy_helper_module.html)


deploy

Code delivery to the production server. Database migration. Zero Downtime Deployment. Ansible, Kubernetes.

Code delivery to the production server



Linux or Mac users sometimes need to run Windows-only programs, and, in turn, Windows users, especially programmers, may need to launch Linux or another version of Windows. The classic example is Photoshop or games.

The most obvious way, and it’s a costly affair, is to buy a second computer. Another way is to install Windows next to your main operating system. Generally, such an installation may lead to a system crash, but once you’ve done you’ll be able to choose a system to boot. But there is a third way, **virtualization**.

- [Virtualization](#virtualization)
- [Hosting](#hosting)
- [FAQ](#faq)
- [Related guides](#related-guides)
## Virtualization

**Virtualization is the creation of isolated environments within a single physical device** (in our case, a computer). Each environment appears to be a separate computer with unique characteristics, such as available memory, CPU, etc. This environment is called a set of logical resources or a **virtual machine**.

*Virtualization allows you to run the operating system like a regular program right on your computer!*

The OS running a virtual environment is called **the host** and the OS running in a virtual environment is called **the guest**.

A special program (in fact, also an OS) called **a hypervisor** creates and manages virtual machines. The hypervisor isolates operating systems from each other and provides protection, security, and resource allocation between running OSs. Depending on the type of virtualization, the hypervisor can work directly with the hardware without a host system, or through the main OS installed on the host machine. In the first case, we use **hardware virtualization**, and in the second case, **software virtualization**. The second is regular for home computers.

Unlike installing two parallel OSs in one machine, virtualization is a much more secure way. You can uninstall and reinstall the systems at any time. You can create as many virtual machines as you need.

### Hardware virtualization

As the name implies, hardware virtualization is related to the hardware, i.e. the processor. In this case, unlike software virtualization, guest operating systems are controlled directly by the hypervisor without the participation of the host OS.

Hardware virtualization is much more efficient than software virtualization, since the hypervisor, unlike the host OS, creates a very small overhead while software virtualization is divided into several subtypes. You can read more about it on [Wikipedia](https://en.wikipedia.org/wiki/Virtualization).

### Container virtualization

**Container virtualization** is another matter. Unlike the previous types, it is not related to running the OS in an isolated environment. With container virtualization, isolation occurs at the OS processes level.

Currently, this kind of virtualization is used only in Linux and is available thanks to two kernel features: cgroups and namespaces. These features make run only one process as if it were running in its own world, with its own network, own disk, own file system, and so on. With this kind of virtualization, one process runs on **the same operating system and the same kernel**. For example, you cannot run Windows on Linux. This virtualization is applied at the level of service that is part of the software product. The most famous projects are OpenVZ, Docker, LXC.

## Hosting

Each virtual machine receives as many resources as you specify. The hosters (companies providing web hosting services) use it a lot. In fact, each user creates his own virtual machine with limits set up by selected plan (memory, CPU, etc.).

In addition, virtualization isolates machines from each other, which means you don't have to worry if users try to harm the system or neighboring users. Such a service is usually called VPS (virtual private server) and is cheap in the basic configuration.

Virtual machines allow you to utilize hardware resources more efficiently. Not all users need all the hardware power that the hoster has, or simply do not want to pay for it. A virtual machine can use only a part of the hardware capacity, which allows hosting dozens of clients on one machine. It's a win-win situation: the user is satisfied, and the hoster earns his money.

## FAQ

### What to do if the CPU does not support virtualization?

It is extremely unlikely, but even in this case it is possible to start a virtual machine. However the performance will be very low, since program virtualization will be implemented instead of software virtualization. In other words, it's better to upgrade your hardware.

{/*-
## Related guides

1. [What is Vagrant](/vagrant/). Vagrant allows creating and configuring lightweight, repeatable, and portable development environments for virtual machines.
2. [How to work with Linux using Windows](/ubuntu-linux-in-windows/). Instructions for installing Ubuntu Linux on Windows using various virtualization technologies.
-*/}


virtualization

What is the point of this technology, and what are the types of virtualization.

Find out what virtualization is and how it works. It allows you to run an operating system like a regular program on your computer.

What Is Virtualization, and Why Do We Need It?


Software development is a challenging process that tends to become much more complex as the number of participants increases. More people in the team create more communication and require more synchronization (sharing knowledge of system parts and processes, keeping track of the business and its requirements). The cost of error increases, and the system can no longer fit in just one head, while a change on one side triggers other changes.

Different teams cope with these conditions in different ways. Some of them keep a high development pace and make regular releases, while others are slowing down significantly: negotiations take more time than development, quality drops, and version updates become stressful and adventurous. The overall release speed of new features by such teams can vary times or even an order of magnitude.

There are many reasons for this crucial difference. Here are some of them:

- Errors of Top management in the core business. When a company does the wrong things, its efficiency does not matter – it will eventually close down. This topic is beyond our current guide
- Errors of Top management in the area of business processes. If there are some issues at this level, everything else is irrelevant. Even the wrong bonus structure can lead to team rifts and ultimately shut down development at the end
- Human factor. Personal virtues and vices can cause problems for the rest of the team or the whole project. The main problem is this part cannot be rectified by any process. It is either a behavior change or a break-up
- Poor development process. This applies to every engineer and includes everything from communication and task handling to testing and code review

Some problems are either difficult or impossible to solve (at the developer level). But others, especially those relating to engineering practices, should be continually improved and fixed. Programmers should be very involved in this.

- [Books](https://en.hexlet.io/pages/recommended-books)
  - Peopleware: Productive Projects and Teams
  - Mythical Man-Month, The: Essays on Software Engineering
  - The Clean Coder: A Code of Conduct for Professional Programmers
  - The Goal: A Process of Ongoing Improvement
- [Manifesto for Agile Software Development](https://agilemanifesto.org/iso/en/manifesto.html)
- [Bus Factor](https://en.wikipedia.org/wiki/Bus_factor)
- [Cargo cult](https://en.wikipedia.org/wiki/Cargo_cult)

Although there are many practices, the net result is how quickly your customers get the results of your work and how pleased they are with it. Below is a checklist to see whether the team is using those engineering practices that are considered most effective.

_Compliance with these standards does not guarantee the company won't have problems. It may be a cargo cult, or its processes may be so formalized that they hinder rather than help. On the other hand, there is an exception to every rule and there will always be some projects which cannot apply some of the following. Finally, some of these approaches may conflict with one's values._

- [Coding](#coding)
- [Development environment](#development-environment)
- [Quality](#quality)
- [Development process](#development-process)
- [New version release (more relevant to web projects)](#new-version-release-more-relevant-to-web-projects)
## Coding

**Good**

- [VCS](https://en.wikipedia.org/wiki/Version_control). The code is put under version control (usually Git)
- [Collective code ownership](https://en.wikipedia.org/wiki/Extreme_programming_practices#Collective_code_ownership). Any member of the team can change any piece of code in the system at any time
- [Coding standard](https://en.wikipedia.org/wiki/Extreme_programming_practices#Coding_standard). All team members adhere to the coding standards adopted for the stack (language, platform)

**Bad**

- Lack of a unified style. Everyone writes code in the style they are used to. There are no common standards at all or there is one but it is absolutely different from the generally accepted rules
- There is no version control. Code backups are used instead, and developers have to negotiate so that they don't overwrite each other's changes
- The code has an "owner". Developers protect their piece of code from encroachment by other team members

**Links**

- [Trunk Based Development](https://trunkbaseddevelopment.com/)

## Development environment

**Good**

- Development environment. Development is executed in a dedicated dev (development) environment. This is usually a local machine (possibly using Vagrant or Docker Compose). Each developer has a unique environment, and changes in one environment can't affect others
- One-click, automated deployment of the environment. This makes it easy to introduce newcomers to your project, to quickly and automatically propagate infrastructure changes and operate with no concern of crashing, as it is easy to rebuild
- Infrastructure as code. Configuration changes are propagated through the project code. One more dev environment deployment (with new project code) is enough to push all updates
- The development environment is as close to a production environment as possible. If service is running on Linux then the development environment is running on Linux as well. The same applies to other aspects

**Bad**

- Deploying and setting up the environment is done either with manuals or by the "try to run, read the error message, google for solution, and fix it" method. It’s expensive and inefficient. Manuals become outdated just after they have been written. A new person can spend days deploying the environment from scratch
- Manual configuration update. All developers receive some directive to make local configuration changes to the environment (e.g., add something new) for the new code to work
- Shared database for all developers. One person's actions can affect overall performance. The accidental crash also slows down everyone else

## Quality

**Good**

- Test coverage. Tests increase confidence in the code performance. Good tests have a positive impact on a code design. Test-covered code is usually better than untested one. Although there is a correlation
- A feature that has been partially tested or hasn't been tested at all is not considered completed. Testing reduces the workload of the rest of the team and positively affects the quality of a subsequent code. Moreover, it's best to write tests immediately, as there is often no time left for them later
- Developer is responsible for his feature all the way to the end. A feature is only considered completed when it is running in a production workflow. Everybody in the team must understand that the most important goal is to deliver value to the customer. As long as no one uses the feature, it doesn't matter if it's written, because the business is getting nothing at that point
- The team members reviewing each other's code (to a reasonable extent). The reviewing process helps both find bugs and learn from each other
- [Pair programming](https://en.wikipedia.org/wiki/Pair_programming). This is an effective approach not only for programmers but also for programmer–tester or novice–expert pairs
- [Continuous integration (CI)](https://en.wikipedia.org/wiki/Continuous_integration). The project repositories are connected to a CI server, where the code style is checked after each commit (by running linters), tests are run, and the project is built (e.g. compiled)
- Do [postmortems](https://www.pagerduty.com/resources/learn/post-mortem-incident-report/) for incidents
- [Retrospective](https://en.wikipedia.org/wiki/Retrospective#Software_development). The process is continually improving, and every team member has an impact changes

**Bad**

- No testing. The testing of new code is done manually by clicking through. This way has disastrous consequences, such as low delivery speed and most likely poor code quality
- No code review. Different code styles, isolation of programmers from each other, limited sharing of experience, poor production decisions
- The programmer considers the feature closed when their code reaches the main branch. New code remains idle and useless and may even become obsolete before it reaches the customer
- [KPI](https://en.wikipedia.org/wiki/Performance_indicator). Heavy use of quantitative metrics: lines of code, released features, closed bugs. Instead of being result-oriented, developers strive to meet KPIs, even if it goes against business goals
- High degree of formalization. Speed slows down, motivation drops

**Links**

- [Extreme programming](https://en.wikipedia.org/wiki/Extreme_programming)

## Development process

**Good**

- Developers are guided by [12factors](https://12factor.net) principles. Applications are easier to deploy, scale and monitor
- A single test runs in a fraction of a second. Test-driven development implies frequent testing during debugging. The speed of launching a particular test is extremely important here – it should be fast enough to keep the developer on track
- Effortless and enjoyable test writing. It is a kind of litmus paper to determine how good the project's test coverage is. If you have to force yourself, likely, the tests are poorly written (e.g. a lot of mocks), and there won't be enough of them
- [Test-driven development (TDD)](https://en.wikipedia.org/wiki/Test-driven_development). If possible, write tests before the code. There are several reasons why this is important:

  Tests require you to think not about the implementation, but about the way the code being tested will be used. Because of this approach, programmers detect flaws in their interfaces at a very early stage.

  The code must be tested in any case. If there is no test, it has to be checked manually.

- Once a bug is detected, first a test is written to reproduce it, then a fix is made. This is the only case where tests really help

**Bad**

- You have tests, but you have to force yourself to write them because they are difficult to write, take a long time to run, often crash or you have to rewrite them all the time
- Running one test takes seconds. Such a test is hard to run in TDD and the total test execution time becomes too long
- The code is fixed directly in production (the place where it works). No comments

**Links**

- [12factors](https://12factor.net)

## New version release (more relevant to web projects)

The production environment is the infrastructure (e.g. servers) where the project is deployed. It provides access to the project for end users.

Deployment is the process of updating the project in the production environment.

**Good**

- Automation. Deployment is automated and performed by clicking one button
- Frequent small releases. Deployment is routine and can be done at any time when features are ready, with no team distractions
- Zero Downtime Deployments. Version updates are transparent to users
- Technically, deployment can be done by any team member (i.e. everything is well automated)

**Bad**

- Manual deployment. For example, by direct management from the server. It is the most unreliable and unscalable approach, prone to error, which can take a considerable amount of time. Doesn't work with multiple servers
- Deployment comes with emotional tension and involves a large number of participants. This atmosphere makes everyone tend to slow down their code deployment, which causes even more problems and hurts the business
- Deployment process takes dozens of minutes or hours. This likely means that the build process is integrated with the deployment. These tasks need to be done separately
- Deployment happens once a week or rarer. The more changes deployed at once, the greater the chance of failure, and the harder it is to track the impact of each feature on business outcomes. In addition, changes that have been made before and were waiting to be deployed to production get overlooked
- Long downtimes during deployment. Users have to wait for a deployment to complete. This situation interferes with frequent deployments
- Deployment is done by a single person. Knowledge is stored in one head. Then this person goes on vacation or getting sick, ruining the whole process, as the rest of the team don't understand how it's all set up
- Deployment of configuration. Updating a configuration that is not related to the code logic (e.g. changing a database password or database address) requires a second deployment. These parameters are strictly infrastructural and should go into the code as described in Twelve-Factor App

**Links**

- [What is DevOps?](https://www.atlassian.com/devops)


check-list-of-engineering-practices

Approaches to testing, deployment, development, and processes.

Essential Software Engineering Practices Checklist for Your Company


- [System installation](#system-installation)
- [Version managers](#version-managers)
- [Universal version manager](#universal-version-manager)
- [Conclusion](#conclusion)
## System installation

To run code in any language, you need to install its interpreter (or compiler). Different operating systems do it differently: some of them use package managers, for example, *apt* or *yum*, and some download the installer directly from the repository. Some languages come preinstalled with an operating system, for example, Python. In particular, Python plays a key role in the Linux OS and its distributions.

```bash
# Ubuntu
sudo apt install nodejs # installs most recent version
```
The standard installation method works well only at the very beginning during the initial setup. Then, over time, different problems begin to surfase. For example, at some point, a new version of the language comes out but the project you are working on requires the latest version. Usually, it takes some time before the language becomes available for installation through package managers. And in this case, you either have to wait, which isn’t always convenient, or look for another installation method. The last one often becomes a challenge and takes a lot of time, with hours of googling and installing additional libraries. All this eventually clogs the system and sometimes breaks it.

Another serious problem arises when developer requires different versions of the same language for different programs. It happens surprisingly often since there are plenty of options in development: various company projects, pet-projects, open source.

*It’s important thing to mention, that all of this doesn’t concern those, who have completely thrown themselves into Docker and Docker Compose. However, even in this case, languages are required to work with open source.*

## Version managers

To solve these problems, the developers came up with version managers. A version manager is a special program designed to handle language versions. With its help it became possible to install the required versions and switch between them. Unlike the package managers that come with operating systems, version managers always allow to install the latest versions of languages as soon as they come out (including installing alpha and beta versions).

For example, you can use [NVM](https://github.com/nvm-sh/nvm) (Node Version Manager) for Node.js:

```bash
# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
# Installation does not mean activation. After installation, the version that was before installation will remain active
nvm install node # Install the latest available version of the node
nvm install 6.14.4 # or 10.10.0, 8.9.1, and so on.
nvm ls-remote # list of available versions
nvm use node # Activate the last installed version of the node
nvm use node 17 # Activate the required version
```
To simplify the work, version managers usually enable users to create a special file within the project that captures the required version of the language. In some cases, version managers track this file and switch versions automatically.
```bash
echo "17" > .nvmrc
# This command spotted .nvmrc file and used the version specified there
$ nvm use
Found '/path/to/project/.nvmrc' with version <17>
Now using node v17
```
In today’s world its hard to imagine a language without version manager. Moreover, some of the widely used languages, such as Ruby, have many competitive version managers:
* go: gvm, g
* java: jabba
* ruby: rbenv, rvm, chruby
* php: phpenv, phpbrew
* python: pyenv

Version managers also solve a few more important tasks. Customarily, when a programmer interacts with a language installed directly, he has to use *sudo* when installing global packages. The fact is that the standard language installation scheme is intended for all users at once. Hence, all the necessary files, including global packages, get into shared directories that require administrative rights. From the security perspective, this is a vulnerability that developers of open source libraries can exploit (and sometimes they do). Version managers install everything in the current user's home directory, where he already has full rights. On the one hand, it allows not to run package installation as an administrator, and on the other hand, the system isn’t clogged. Version manager makes it painless to remove a language and all its packages. All you have to do is delete the directory (although it’s better to do this using the version manager tools).

## Universal version manager

While solving some problems, version managers also cause a few others. Firstly, there are too many of them, so their popularity constantly changes. Secondly, the process of installing the version manager can be more complicated than installing the language. The problem is that they need to be universal and work everywhere, which is extremely problematic, considering the diversity of modern ecosystems. It’s enough to look at the size of the NVM documentation to assess the scale of the disaster. Thirdly, all these managers work differently and have different commands. It complicates the process of switching between them while working with various languages.

All this led to the next logical step. Eventually, a universal manager [asdf](https://asdf-vm.com/) appeared on the scene, and, thanks to plugins, is now able to work with any language. A shortlist of its advantages:

* A single command line utility to work with all languages
* A single interface for all languages
* Automatic switching between languages within each project
* A simple plugin system that allows to add any language

Now *asdf* has become quite popular and is gradually replacing every other version managers (technically, language-specific managers are more often used there). It has a bit more complex command system due to the need to support many languages, but otherwise it significantly simplifies the whole process.

```bash
# asdf has excellent documentation, which clearly shows how to install it,
# and what dependencies can be required in different systems
# Installing
git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.9
# Restart the terminal after that
echo '. $HOME/.asdf/asdf.sh' >> ~/.bashrc
# To work with a particular language, it's necessary to install the respective plugin
# The list of available plugins is on the project website
asdf plugin add nodejs
# Language installing
# Instead of nodejs, you need to substitute the name of the plugin you are working with
asdf install nodejs latest # latest means the latest version of the required language
# Installing the required version
asdf install nodejs latest
# Installing the required language version by default
asdf global nodejs latest
# Shows current language versions installed through asdf
asdf current
elixir         1.10.1-otp-22 (set by /Users/user/.tool-versions)
erlang         22.2.7   (set by /Users/user/.tool-versions)
nodejs         17.0.0   (set by /Users/user/.tool-versions)
php            7.4.5    (set by /Users/user/.tool-versions)
python         3.8.2 2.7.16 (set by /Users/user/.tool-versions)
ruby           2.7.0    (set by /Users/user/.tool-versions)
yarn           1.22.4   (set by /Users/user/.tool-versions)
```
## Conclusion

Working with different versions of the language is a challenge, which is made easy due to version managers and Docker (for advanced users). Among all managers, *asdf* stands out and at the same time becomes a universal tool for managing any language and even regular programs.


version-managers

How to manage different language versions

The right approach to installing programming languages and updating them. Universal asdf version manager.

What Is a Version Manager?


**One of the most important skills for a programmer is the ability to find answers and solve problems using Google. A lot of the problems that beginners face have already been solved and described before. All you need is to learn how to find these solutions and answers.**

- [Websites](#websites)
- [Language](#language)
- [Search engine](#search-engine)
- [Query language](#query-language)
- [Library search](#library-search)
- [Search by error message](#search-by-error-message)
- [Behavior search](#behavior-search)
- [Official documentation](#official-documentation)
## Websites

The main sites to get your questions answered:
* [GitHub](https://github.com)
* [Stackoverflow](https://stackoverflow.com)

## Language

As practice shows, at the beginning most programmers try to search for information in their native language. Sometimes you can find the answer this way, but more often — no. **The universal language for programmers is English** – the whole world speaks it. The amount of information in the English-speaking segment is noticeably higher than in any other. Besides it’s a lot fresher. Learn to formulate your thoughts more accurately, and while searching - compose a set of words in English. This way you will master the terminology faster.
<Banner name="site-code-basics" />

## Search engine

To continue the previous section it’s important to note that the best way to search for information is using Google. Although local search engines are fine for certain tasks, the English-speaking segment isn’t their main market, and they are much inferior to Google. For example, you will notice that after a while Google adapts to your queries and starts showing more relevant links. It can understand which programming language you prefer and will output the related answers.

Another important point: Google search works even better than site search. If you need something, for example, on Github, it's better to make the proper Google query to get a better and faster result. The below section "query language" provides further details.

## Query language

Each search engine has a query language. It includes special search operators allowing you to specify more precisely what you want. Here are a few common search techniques:

* `site:stackoverflow.com how to test react code` — this search will be performed through [Stackoverflow](https://stackoverflow.com/).
* `add class to element -jquery` — a hyphen denotes stop words, therefore, all variants excluding these words will be found.
* `"immutable js"` — double quotes indicating to look for an exact match.

The full list can be found [on the Google support site](https://support.google.com/websearch/answer/2466433?visit_id=1-636424030566191968-2246914586&p=adv_operators&hl=en&rd=1).

## Library search

The vast majority of libraries are located (they say, "hosted") on [GitHub](https://github.com). Let’s say you need to find a library for executing HTTP requests in JavaScript. To do this you can form the following request: `github js http client`. Google will show you a dozen links to different repositories. Of course, you can also use the query language: `site:github.com js http client`, but generally it's enough to simply indicate `github`.
The same search strategy can be used for well-known libraries: `github express`.

## Search by error message

Before you search by error message, it’s necessary to understand where *the error message* is here exactly. Although the output contains a lot of relevant information, it’s not a description of this error. For example:

```shell
There was 1 failure:
1) App\SolutionTest::testResult with data set #2 (0, 2, 2, 1, 2)
Failed asserting that '1' matches expected 0.
/usr/src/app/tests/App/Tests/SolutionTest.php:15
FAILURES!
Tests: 3, Assertions: 3, Failures: 1.
Makefile:2: recipe for target 'test' failed
make: Leaving directory '/usr/src/app'
make: *** [test] Error 1
```

The output has a lot of text but there is only one real error message: `Failed asserting that '1' matches expected 0.`. In this particular case, it’s approximately clear what is wrong and where to look (the file and line are indicated in the stack trace below). But so happens not always. If you correctly identified the error message, you may also want to do one thing. Often such messages are personalized: they take specific values of parameters that relate to your environment. For example, paths to files. Hence, if you search throughout the error message, then most likely Google won’t find anything. For example, in the message above, such parameters are `'1'` and `0`. If you clear the phrase, there will be `Failed asserting that matches expected`. That’s what you have to look for. You can also add language name: `php Failed asserting that matches expected`.

## Behavior search

Sometimes an error message is either missing or cannot lead to a correct answer (since it’s a consequence, not a cause). At this point, you need to be creative and make a sentence in English. A set of keywords will also work. If the search wasn’t successful then try to add `site:stackoverflow.com`. Stackoverflow is a unique place having answers to almost all similar questions.
If you know which library or program exactly the error belongs to, it would be useful to find its repository on Github and study the Issues section. If there definitely is an error, and it’s relevant, then most certainly someone has already mentioned it.

## Official documentation

Search is good but don’t forget to look into the official documentation for tools (including programming languages). Official (and not only) documentation, as a rule, is divided into following types:

1. [Getting Started](https://guides.rubyonrails.org/getting_started.html) is a small (not always) step-by-step guide helping to create a minimal working version and quickly start working with the tool to see it in action. It’s the first document you need to look for.
1. [Guides](https://laravel.com/docs/5.5/routing) are descriptions of the tool components. They are written in a narrative form, so it's not hard to read them from start to finish. They make it easier to learn large blocks of information.
1. [API](https://bit.ly/2uq98XM) is a concise documentation on all possible application functions. It is more intended for finding answers to specific questions.
1. [Tutorials](https://blog.codeship.com/an-introduction-to-apis-with-phoenix/), in contrast to quides, are aimed at various options for using the tool.


how-to-search

Asking the right questions is half the battle.

Most of the problems that beginners face have already been solved and described. You just need to learn how to find the solutions and answers you need.

How to Search for Technical Information


There are no programs without errors. Their number can be reduced with the great help of a type system, linters, tests, or even a whole department of testers, but it’s impossible to remove them completely. This is the reality we must face; the best thing we can do is learning how to track these errors and fix them as soon as possible.

- [How you shouldn’t work with errors](#how-you-shouldn’t-work-with-errors)
- [How to work with errors](#how-to-work-with-errors)
- [Conclusion](#conclusion)
## How you shouldn’t work with errors

It's quite common in development: programmers write some code, release a new version of a website or application and move on. In the meantime, some users start experiencing problems: something freezes or crashes, issues with sending forms out, data being displayed incorrectly, and so on. This may last long enough for someone to lose patience and finally complain to the support service. Then, along the chain, it makes it to the programmers, and they try to find the root cause behind the error, when it began or who caused the bug to happen. The clarification of all the details begins, and most likely with the participation of users who are willing to help. With this approach, a huge number of errors remain unnoticed for a very long time, and the worst part is that users leave. Can we avoid this? Clearly, we can.

## How to work with errors

There are plenty of services called Error Trackers. They collect information about occurring errors in real time and notify the development team. These services integrate with a multitude of available platforms, from televisions to mobile applications and websites (both frontend and backend).

![Rollbar Dashboard](/assets/images/error-tracking/rollbar-dashboard.jpg)

Above is a screenshot of a dashboard service [rollbar.io](https://rollbar.io/) which we use for all our Hexlet projects. You can track the frequency of critical errors in the last 24 hours for each project. This graph with performance metrics helps you diagnose issues immediately. Below is the output of specific project's errors. According to the icons, most of the errors are caused by JavaScript.

![Rollbar Project](/assets/images/error-tracking/rollbar-project.jpg)

Each such service provides libraries for different languages and platforms, they are usually added to the code and called on in case of errors. These libraries send not just the error but also additional useful information about the environment. Such information may include data about users, their browsers, application settings, and more.

Ideally, that kind of library is already [integrated](https://docs.rollbar.com/docs/rails) into a framework, for example, Rails. Then you hardly have to configure anything. You just connect the library to the framework as a plugin and it will start detecting errors by itself, without extra interaction. If there is no such integration, well.. then you'll have to write some code to link your application to the library. You can find more information on how to do this in the service documentation. [Here](https://docs.rollbar.com/docs/react) is an example of Rollbar integrated into React. Once the connection is successfully established, the detected error will look like this:

![Rollbar Error](/assets/images/error-tracking/rollbar-error.jpg)

If we take a closer look at the top menu, we can see that tabs provide very useful information about errors.

But detecting an error is only half of it. You then need to notify the team somehow without flooding everyone with spam. The thing is that errors usually don't occur just once. If it’s a common error and the number of users is high, then you can easily catch one error thousands of times per minute. And if there is a notification for each occurrence (by email or in Slack), then such a service wouldn’t work for long.

![Rollbar Error](/assets/images/error-tracking/rollbar-notifications.jpg)

That’s why such trackers work more effectively. When the error occurs for the first time, the service notifies the development team with real-time alerts so that they can quickly address it. If it happens once again, then no more alerts. At least not every time. For example, alerts can be sent at the first, tenth, hundredth, thousandth and so on as they occur. This is the first part. Further, after updating the application version, trackers label errors as “corrected”. This makes it much easier to track errors that developers either forgot to fix or did not fix properly. Most often, alerts start pouring in after the deployment. In order to prevent future issues, you need to notify the tracker service about deploys. You can read more about this in the [documentation area](https://docs.rollbar.com/docs/deploy-tracking) of the relevant tracker.

## Conclusion

Error trackers are very important tools, and it’s hard to imagine production without using them. At that one of many services or specialized software can act as an error  tracker (for example, [Sentry](https://github.com/getsentry/sentry)), a service installed on servers in case of high security requirements.


error-tracking

A fast way to track errors in a working application

Errors are unavoidable, so you should be able both to track and prevent them. There are services that can help manage this task.

What Is Error Tracking?

In a nutshell, (**A**pplication **P**rogramming **I**nterface) refers to how programs or services can request data from other services.

- [Twitter API Example](#twitter-api-example)
- [HTTP API](#http-api)
- [Non HTTP API](#non-http-api)
## Twitter API Example

Twitter has a website where we can view tweets. By clicking on the links, we can go to addresses where different information is available, such as your feed, a page for a particular tweet, your profile, password reset, your advertising account, and so on:

* https://twitter.com/hexlet_io – a particular user's page
* https://twitter.com/hexlet_io/status/1613590336932122643?s=20 – a page for a particular tweet

These pages return HTML that presents the information as we see it, with all the design elements. Twitter's developers adapted the site for small screens, so it opens from any device with a browser. But besides the website, Twitter has a mobile application that displays the same data, but with a different, more convenient interface adapted for that specific phone.

The mobile app, unlike the website, is physically stored and runs directly on that specific phone. And the whole appearance is also defined inside the application. For such an application to work, you need clean data, but that data aren't on your phone, they're on Twitter's servers. How do you get them on your phone? Ordinary page addresses don't suit us, as they don't return the data we need but rather ready HTML pages.

It is where the API comes to the rescue. Twitter provides HTTP addresses where pure data, rather than specific web pages with ready-made appearances, are available. Data at these addresses are given in a structured format, most often JSON. One service packs the data into JSON format, then another service unpacks it from JSON and uses it to display the data in the app. Here you see an example of a user request:

```shell
# The address to which the list of specified users is returned
# In this case, one user is requested
https://api.twitter.com/2/users/by?usernames=hexletHQ

# As a reply, you get some JSON of the following structure:
{
  "data": [
    {
      "created_at": "2013-12-14T04:35:55.000Z",
      "id": "2244994945",
      "name": "HexletHQ",
      "description": "Hexlet - programming courses",
      "pinned_tweet_id": "1464165859761573893",
      "username": "hexlethq"
    },
  ],
}
```

The task of this API is to provide convenient access to Twitter data. And this API is used both by Twitter in mobile applications and by third-party services that manage Twitter. For example, marketers use services that automatically publish tweets according to a schedule.

By the way, Twitter uses its own API, including widgets. We can embed it on any website to display a particular tweet or feed of tweets, like on [Hexlet main](https://hexlet.io). And, Twitter's site itself runs on the API. It loads JS code into the browser, which implements the logic of the output and goes to the server to get data (via the API).

Twitter has [a site for developers](https://developer.twitter.com/en/docs/twitter-api) with a detailed description of their API, addresses, parameters that can be transferred, and response formats. Similar descriptions are available for any service with a public API, services available for use on the Internet. Of course, a public API doesn't mean a free one. An API can be and often is paid.

![Twitter API](/assets/twitter-api.png)

## HTTP API

The Twitter API is an example of an HTTP API; it works over the HTTP protocol. Browsers use it to load and display sites. HTTP is the most common and convenient way to make an API for public internet services. The whole Internet is flooded with it. It's well-supported by most programming languages. After all, it's a simple protocol that all web programmers know one way or another.

But not all HTTP APIs are the same. HTTP leaves a lot at the mercy of developers. So, HTTP APIs can differ drastically, for example, in their data format. The most popular formats are JSON and XML. They are very similar in appearance, but XML describes data rather than how the data are arranged and appear:

```xml

<Contents>
    <Key>europe/france/cannes.jpg</Key>
    <LastModified>2010-02-17T22:11:12.487Z</LastModified>
    <ETag>"53fc311c15eda0a031809982ccf92aac"</ETag>
    <Size>5061631</Size>
    <StorageClass>STANDARD</StorageClass>
</Contents>
```

There are other formats, but developers use them much less frequently. Generally, HTTP doesn't limit the format, so you can create and use your own.

Besides formatting, HTTP APIs differ in how their addresses are structured. For example, code-basics has a different address for each type of data:

* https://code-basics.com/api/languages.json – list of languages that have coureses
* https://code-basics.com/api/languages/12.json – course information
* https://code-basics.com/api/languages/12/lessons.json – list of language course lessons

This kind of API is often called a REST API. You can read more about it in another article. Sometimes, API addresses can look the same, but different request parameters are sent to it. This method has many variations, such as json-rpc, graphql and others:

```shell
# All requests go to only to this address
https://some-site.com/api/rpc
https://some-site.com/api/rpc?param=value
```

Finally, we have the WebSocket API, where communication goes both ways in real-time, as opposed to the examples above, and the client (the one using the API) has to request the data itself. Websocket is used to make real-time applications such as chat or games. In these applications, data exchanged on the server must be sent as quickly as possible to all interested clients.

## Non HTTP API

APIs don't just have to be HTTP. Typically, these APIs are found within a service to facilitate components interacting. Each part behaves like a small service. These APIs can:

* Use another protocol, such as TCP
* Lean on other approaches, such as queue systems or threads (grpc, rabbitmq, kafka)

http-api

How services interact with each other. Understanding HTTP, JSON, XML technologies


What's an API?


Have you ever wondered how the browser understands which page to open when you enter the site address in the URL bar? This is a much deeper issue than you might think. It needs to be solved in a way that isn't based on moving between sites, but on how computers are connected with each other.


In the 70s, 80s, and 90s, there was a network called ARPANET. It was an attempt by the US Department of Defense to link multiple computers together, so that they could transfer information to each other during wartime. The most important element of this approach was to be able to rapidly transmit data over long distances. Subsequently, the principles of ARPANET formed the basis of the modern Internet.

Initially, the entire network combined computers in four different US institutions:

* University of California, Los Angeles;
* Stanford Research Center;
* University of Utah;
* University of California, Santa Barbara.

The scientists at these institutions quickly came to a consensus that it was more convenient to transfer information about research to each other using the new network. In order to do this, it was enough to know the ID of the computer that the message was being transmitted. These identifiers are now called IP addresses. Every device on the Internet has this identifier, and this is what's used by devices to address each other.

At the very beginning, there were several dozen computers connected to the network, and their IDs were easy to remember. It was possible to write down these addresses in a notebook and use it in the same way as one would a phone book.

Time passed, and by the mid-80s, instead of several dozen computers, the network began to number several thousand. And each of them had a unique identifier, which became increasingly difficult to enter manually or remember. A system was needed that would humanize computer names and store all addresses in one place so that every computer on the network would have the same set of every identifier.

- [hosts file — the first step to creating the DNS](#hosts-file-—-the-first-step-to-creating-the-dns)
- [How the DNS works on the internet](#how-the-dns-works-on-the-internet)
- [DNS resource records](#dns-resource-records)
- [Example of real DNS records](#example-of-real-dns-records)
- [Conclusions](#conclusions)
- [Additional resources](#additional-resources)
## hosts file — the first step to creating the DNS

To solve the problem, the developers decided to use a dictionary that linked the unique name and IP address of each computer on the network. This file was a dictionary, called hosts.txt, and it was responsible for binding IP to computer names. The file was stored on the server of the Stanford Research Institute, and network users regularly manually downloaded this file to their computers to keep the dictionary up to date, because new computers appeared on the network almost every day.

hosts.txt looked (and still looks) like this:

```
192.168.10.36           MIKE-STRATE-PC
Network (IP) address    Computer name
```

Having stored this file on the computer, there is no need to remember the numbers, but the latin name "MIKE-STRATE-PC" can be used instead.

Let's see what the file looks like and try to add a new name there to connect to the computer using this name. To do this, edit the hosts file. You can find it on your computer at the following address:

* On Unix systems: `/etc/hosts`
* On Windows systems: `%Path to the Windows folder%/system32/drivers/etc/hosts`

[![asciicast](https://asciinema.org/a/FOqRvmFFMQPtruCnr59HEGAY3.svg)](https://asciinema.org/a/FOqRvmFFMQPtruCnr59HEGAY3)

We specified the name "MIKE-STRATE-PC" for the computer with the IP address 192.168.10.36, which is located inside the local network. After that, you can use the ping command, which will send a special request to Mike's computer and wait for a response from him. It's like you're knocking on the door or ringing the bell to find out if anyone's home. Such a request can be sent to any computer.

[![asciicast](https://asciinema.org/a/8H3jKqFsVUNyoxtzn7AWPJ1GH.svg)](https://asciinema.org/a/8H3jKqFsVUNyoxtzn7AWPJ1GH)

With the development of the network and the constant appearance of new customers, this method became inconvenient. All users were increasingly required to download the latest version of the file from the Stanford Research Institute server, which was updated manually several times a week. To add new versions, you had to contact the institute and ask them to add new values to the file.

In 1984, Paul Mockapetris described a new system called DNS (Domain Name System), which was designed to automate the processes of collating IP addresses and computer names, as well as the processes of updating usernames, without the need to manually download a file from a third-party server.

## How the DNS works on the internet

Nowadays, the internet is everywhere, we use it for both mobile and desktop devices. Video surveillance systems, speakers, and even kettles interact with each other via the Internet, and in order to communicate with them properly, we need a system through which users can connect to the desired service using just one request from the address bar. All this falls on the shoulders of the DNS system, which stores a lot more information inside itself than just the IP address and the name of the devices. DNS records are also responsible for sending emails correctly, linking different domains and domain zones to each other.

The DNS is a [distributed system](https://en.wikipedia.org/wiki/Distributed_database), meaning it has multiple nodes, each of which is responsible for its own zone. This is possible due to the fact that the DNS structure itself is hierarchical, i.e., it allocates areas of responsibility, where each parent knows about the location of its child server, and about its area of responsibility.

![dns, hierarchy](/assets/images/dns/hierarchy_en.png)

Let's take a closer look at how the DNS and its components work.

### Terminology

The main components of the DNS are:

**Domain** — the symbolic name for a server on the Internet. Domain names are a hierarchical structure in which each level is separated by a dot. The main levels are:

* Root domain. It's not used in the URL, but it's always implied. All the URLs on the Internet start from this.
* Top level domains. These include .com, .net, .es, .org and so on. This domain is also called the first level domain.
* Second level domain (or main domain). This is the main name of your site.
* Subdomains (third, fourth, fifth, etc. level domains). This includes all subdomains of the main domain.

**The DNS server** — the system responsible for storing and maintaining up-to-date records of its child domains. Each DNS server is only responsible for its own zone, i.e. the .io DNS server knows where the hexlet domain is located, whose DNS server knows the location of its subdomains.

**Root DNS-server** — the system that knows the location (IP addresses) of DNS servers of top-level domains.

**A resource record** — is a unit of information on a DNS server. Each resource record has several fields:

* Name (the domain to which the record belongs)
* Type
* Parameters
* Value

### Connection

It should be understood that the domain name is just an abstraction for people. The computer itself and applications (e.g., the browser) only access services within the Internet via IP addresses.

Let's look at the process of obtaining an IP address by domain name using the domain `en.wikipedia.org`.

Two variants of events are possible:

1. The computer sends a request to a DNS server it knows. Generally, it's the DNS server of the Internet service provider (ISP): *What is the IP-address of the domain en.wikipedia.org?*. The ISP DNS server finds information in its database stating that the domain `en.wikipedia.org` is located at the IP address 208.80.154.224 and returns the value to our computer. This process is similar to how the `hosts.txt` file was used.

2. The nearest known DNS server has no record of the IP address where `en.wikipedia.org` is located. In this case, a chain of processes is started which will give our computer a domain IP address:

    * Since the domain is a hierarchical structure, and all DNS servers know the IP addresses of the root DNS servers, so they get a request to provide the domain IP address.

    * Root DNS servers are aware of where the DNS-servers of top-level domains are. These addresses are returned to our provider's DNS server, and after that, a request is sent to the required DNS server (in our case, to the DNS server of the .en domain) to obtain the IP address of the en.wikipedia domain.

    * In accordance with its area of responsibility, the DNS server of the top-level domain returns the IP address of the wikipedia domain's DNS server, which gets a request for the en subdomain's IP address.

    * The DNS server returns the IP addresses of en subdomain, and then our provider's DNS-server of returns this address to our computer, which is then able to access the domain en.wikipedia.org by its IP-address.

### Recursion in DNS

You can see that both variants described above are very different: in the first instance, we just sent a request and got an answer, while in the second, we had to go from the root domain itself in the search for the record we needed. This process is recursive because the nearest DNS server continuously sends queries to other DNS servers until it gets the necessary resource records. This process can be visualized as follows:

![dns, structure](/assets/images/dns/structure_en.png)
<small>For queries 1 and 2, the nearest server will receive information about the location of the DNS servers within the area of responsibility of the server to which the request was sent. Query 3 will retrieve the necessary resource records for the wikipedia domain and its subdomains.</small>

Recursive searches are quite a long operation, are also a heavy load on the network and the DNS servers. In order to avoid recursion, each DNS server [caches](https://en.wikipedia.org/wiki/Cache_(computing)) information about records it receives, so that they can quickly return this information to the user.

As you can see, recursive search involve finding the final answer to our query by searching through all the necessary DNS servers, starting from the root one. In contrast to this method, there is also an iterative query, which, unlike the recursive one, performs only one iteration; a query to the nearest DNS server, from which we can get both the cached response and the data from the zone it's responsible for. It is important to note that an iterative query involves only one query.

Generally, DNS servers are able to send recursive queries, because in these cases, the response can be cached, which then will reduce the load on both the server itself, and on other DNS servers. The period that the DNS server caches information for is set in the DNS resource record, which is what we'll be talking about now.

## DNS resource records

The modern internet involves not just obtaining an IP address by domain name, but also forwarding e-mails, connecting additional analytics services to sites, and setting up secure HTTPS protocols. This is most often done by using DNS resource records.

Let's look at what resource records are used and what they indicate. The main DNS resource records are:

**A records** — are one of the most important records. This is the entry that points to the server IP address, which is tied to the domain name.

**MX records** — indicate the server that will be used when sending domain email.

**NS records** — point to the domain DNS server.

**CNAME records** — allow subdomains to duplicate the DNS records of their parents. This is done to redirect requests from one domain to another (most often to redirect a domain with a www subdomain to a domain without this).

**TXT records** — store text information about domains. They're often used to confirm ownership of a domain by adding a certain string that sends us the Internet service.

Resource records are almost always the same, but for some records, other fields may appear, for example, MX records also have a priority value. Generally, resource records have the following structure:

```
Record name   TTL   Class   Type  Value
```

Let's go into more detail:

**The record name** — specifies the domain the resource record belongs to.

**TTL** *(time to live)* — the time in seconds that the value of the resource record will be cached for. This is necessary to unload DNS servers. Caching means that the nearest DNS server is able to know the IP address of the requested domain.

**Class** — it was assumed that DNS can work not only on the Internet, so the record also indicates its class. Currently, only one value is supported - IN (Internet).

**Type** — indicates the type of resource record; we discussed the main ones above.

**Value** — means the value of the resource record itself. Depending on the type of resource record, values can be presented in different ways.

Let's see what form these records are stored in on DNS servers using the domain google.com as an example. To do this, we'll use the dig utility, which gets all available resource DNS records from the DNS server and displays them to the user.

The dig utility is a DNS client and is part of one of the most common BIND DNS servers.

[![asciicast](https://asciinema.org/a/vbZ2Tcy7Wb3z9LLE3rEupGxfZ.svg)](https://asciinema.org/a/vbZ2Tcy7Wb3z9LLE3rEupGxfZ)

## Example of real DNS records

![dns, output](/assets/images/dns/output_en.png)

Don't be intimidated by the long output. Most likely you can understand almost everything shown here. Let's analyze the output of each section in more detail.

The output consists of several parts:

* Header
* Request section
* Response section
* Service information

**Request header**

```
; <<>> DiG 9.10.6 <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57684
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
```
Here you can see the flags of our request, the number of requests and responses, and other service information.

**Request section**

```
;; QUESTION SECTION:
;google.com.			IN	A
```

The request section specifies the domain being accessed, the record class, and the records we want to retrieve. ANY indicates that all available resource records need to be output, but if you want to experiment with the utility yourself, you can use a special key to output only specific records.

**Response section**

The response section is quite large, so for convenience let's break it down by resource record type.

```
;; ANSWER SECTION:
google.com.		30	IN	A	74.125.205.100
```

The A record points to the IP address bound to our domain.

## Conclusions

DNS servers now form the backbone of the entire internet and are used in almost every action a user takes, whether it's going to a website, sending an e-mail, using app on the phone, and so on. Therefore, knowledge about the principles of operation of DNS servers and basic resource records, thanks to which it is possible to navigate the Internet, are important for the developer.

## Additional resources

* [How DNS works comics](https://howdns.works)


What is DNS, how it works on the Internet, what are the resource records (explanation and examples)

What's a DNS server in simple words?

docker

Docker is a program that allows the operating system to run processes in an isolated environment based on specially created images.

How do you build a project infrastructure, and how do you bundle and run services? We'll discuss isolating processes in the operating system without relying on heavy virtualization.

Guides on technologies, tools, and practices