7 July 2021

Deploying app in a linux server

by Hasan

Spinnig up new Linux server and tightening its secuirity with SSH keys and firewalls
Using Linode Deploying on Linux Server
Cloud.linode.com
Select operating system and Region (near to the app)
Select plan
select root password
Select or not (Backup or Private IP)
Create and wait for all configuration done
Go to networking tab
- SSH Access –> machine address IP address will be available
  Entering first on the Server
- ssh root@ip-address will be the ssh command
- command line (bash shell in my case) copy the previous line ssh command
- Ask root password, that was previously selected by us.
- after entering the password you will be at linode server.

## Setup the server

apt update && apt upgrade
hostnamectl set-hostname flask-server # actully setting hostname. In our case we have set the hostname as flask-server
Need to open a file with an editor. we will use Vim. So vim /etc/hosts.
- We will see there 127.0.0.1 localhost. After this line we will write the ip address of server machine and then enter tab and write host name. Example 45.33.123.214 flask-server
- Save the file

Adding new user

adduser hasan
maybe ask for password and we can set the password for this user.
Now ask some information for the user. Optional
Now need to add this user to add sudo group, sothat this user can run admin command.
- adduser hasan sudo

logout
ssh hasan@45.33.123.214
ask for a password. Need to enter this user password. Actually we need to write ssh username@ip_address of the server machine and password of this username
Now we are and logged in as a new user

with our home directory we wiil make a .ssh directory. mkdir .ssh
Now open a bash window (if available, otherwise putty or other) in your local computer. Insert this command ssh-keygen -b 4096. If available they will ask you to overwrite or not. If you want you can overwrite. If this is first time then just ask for passphrase. You can make it empty, however if you want more secuirity then you can enter a passphrase. Now we got two extra file. id_rsa and id_rsa.pub(pub means public)
We actully need to move this publich key to our server. scp ~/.ssh/id_rsa.pub hasan@45.33.123.214:~/.ssh/authorized_keys (scp source desitination [in our case destination is username@ipaddress:filelocation in server])
Now check in the server whether the file is available or not. ls .ssh .
Now change the permissions sudo chmod 700 ~/.ssh/ , then ask for the password.
Again sudo chmod 600 ~/.ssh/* . For all files in ssh directory
Now we should be capable of login wihtout password. Check it. try to logout. exit
In local machine bash shell ssh hasan@45.33.123.214. Now no password should be asked.

Now update ssh config file in server
sudo vim /etc/ssh/sshd_config . A password will be asked (because of sudo)
We need to change two values of this configuration file
PermitRootLogin no
__PasswordAuthentification no __
This two values need to be changed like above. Now we need to restart ssh server. sudo systemctl restart sshd

Deploy a firewall

sudo apt install ufw. ufw –> uncomplicated firewall
sudo ufw default allow outgoing
sudo ufw default allow incoming
Now we need to allow certain rules to allow somthing which is necessary for us (ssh,http or certain port)
sudo ufw allow ssh
sudo ufw allow 5000 (5000 port is the port where we have created our flask application. So we need to test before we want to deploy it)
Before allowing port 80 we need to make sure everything is working properly.
Now we need to enable everything what we just set.
sudo ufw allow 5000. Maybe will ask command disrupt existing connection. We will allow it
You can always see what is allowed and what is not allowed. sudo ufw status

Get Application to the server

Mutiple ways (git, ftp), but we are using command line for copy so we will use the same.
Go to our local command line bash window.
If we are using virtual environment, we need to create a requirement.txt file for all our libraries and versions.

Creating requirement.txt

pip freeze > requirement.txt
Move this requirement.txt from home directory to project directory (In our case it is flaskblog directory)

Copy project to server

scp -r Desktop/Flask_Blog hasan@45.33.123.214:~/ (r means recursive means all child folder also. then source destination here we are using ~ means home directory of server)
Go to server and check whether the file is there or not
ls . It should be there

Create Virtual Environment on server

sudo apt install python3-pip
sudo apt install python3-venv (This will allow us to create virtual environment)
python3 -m venv Flask_Blog/venv (This should create a virtual environment). Here Flask_Blog is the directory where we want the virtual environment and venv is the name of the virtual environment
Now check whether the virtual environment is created or not
Now to activate it we need source venv/bin/activate
Now to install the requirement.txt we need to run pip install -r requirement.txt (after -r we need to set the path of our requirement.txt file, here we have in the same directory therefore only requirement.txt is used)

Create Configuration file with all of our environment varaiable

There are different was to do this. If python environment is activated then we can just run python in bash
os.environ.get(‘SECRET_KEY’)
os.environ.get(‘SQLALCHEMY_DATABASE_URI’)
os.environ.get(‘EMAIL_USER’)
os.environ.get(‘EMAIL_PASS’)
Now go to server and create a file sudo touch /etc/config.json
Now edit the file sudo vim /etc/config.json
We will create the json file
- { “SECRET_KEY”: “579……”, “SQLALCHEMY_DATABASE_URI”: “sqlite:///site.db”, “EMAIL_USER”: “hasanme@gmail.com”, “EMAIL_PASS”: “password of email” }
Now save the file

Now goto our application where config.py is available. Open it

   import json
   with open('/etc/config.json') as config_file:
       config = json.load(config_file)

replace os.environ.get to config.get
Now save the file

Run the flask app inside the server

Upto now we have used python run.py. However we want to deploy it to use 0.0.0 that means open to outside world. Therefore we want to run it using flask run.
To run it flask run we need to create an environment variable
To create environment variable export FLASK_APP=run.py in the command line. Means we are creating an environment variable
flask run –host=0.0.0.0
Now we want use our ip address means server ip address. In our case we used 45.33.123.214 and also port needs to used. So we will use 45.33.123.214/5000. Now our app should be available
Do some testing. Everything is working or not. Create some post, Delete some post, Edit some post, create new user, and so on..

Run the application with nginx and gunicorn

Now kill the app from command line. control x
Now go to home directory. cd
Now run bash sudo apt install nginx pip install gunicorn
Make sure that you are still in our virtual environment.
Now need to change some configuration with nginx and gunicorn

nginx and gunicorn configuration

ngingx will be our webserver and it will handle the request, such as static file like that.
It will not handle the python code, gunicorn will do that

Now remove nginx configuration file

sudo rm /etc/nginx/sites-enabled/default

Now we are goind to create a new file

sudo vim /etc/nginx/sites-enabled/flaskblog

Now we will write something in the file

server {
       listen 80;
       server_name 45.33.123.214;

       location /static {
                 alias /home/hasan/Flask_Blog/flask_blog/static;   
       }
         
       location / {
                 proxy_pass http://localhost:8000;
                 include /etc/nginx/proxy_params;
                 proxy_redirect off;
       }
}

Save this file. What here is doind is actually telling where the location of the static folder, in our case all our static files are in the folder where we told after alias name
Second location we are saying that if we are going to our ip address this will happen. Here localhost is server, write now local host is running on 8000 port
Some extra varible for that proxy is supplied by next two lines
Although we have said that we are listening to port 80. But still we donot created it
```
sudo ufw allow http/tcp
```
Since we are done with testing therefore will disallow the port 5000

sudo ufw delete allow 5000

sudo ufw enable

So we want to restart nginx server

sudo systemctl restart nginx

So nginx is done. Now we need to gunicorn to handle python code
```
gunicorn -w 3 run:app
```
-w means number of workers. Then run is the file where our application is situated here(run), then colon and then variable name of our application here (app)

In gunicorn website it is said number of workers = (2 * number of cores) + 1

In linux how many core is available to see.

nproc --all

To run gunicorn one should go to the directory
```
cd Flask_Blog/
```
now run gunicorn
```
gunicorn -w 3 run:app
```
Now if we open we will see it is working
Now we need something which will monitor gunicorn, that means if it crashes it will restart again and will handle different thing to run it continuosly

Monitor gunicorn

sudo apt install supervisor

Now we need to creae a configuration file for sudo

sudo vim /etc/supervisor/conf.d/flaskblog.conf

In the configuration file we will write following line

[program:flaskblog]
directory=/home/hasan/Flask_Blog
command=/home/hasan/Flask_Blog/venv/bin/gunicorn -w 3 run:app
user=hasan
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
stderr_logfile=/var/log/flaskblog/flaskblog.err.log
stdout_logfile=/var/log/flaskblog/flaskblog.out.log

Atfirst where the program is available, then the command to run. Normally our virtual environment is not activated, therefore we should use whole path where the program is available. Then previously described file:varialbe and other configuration for running swiftly. Also logfile directory is added. We did not create it yet. But we will create them

So create those logfile

sudo mkdir -p /var/log/flaskblog

sudo touch /var/log/flaskblog/flaskblog.err.log

sudo touch /var/log/flaskblog/flaskblog.out.log

Now we need to restart supervisor

sudo supervisorctl reload

Check everything
One needs to know nginx has some default thing which can be different from our test, like previously we handled the large picture , but now nginx will not allow it
Normally you don’t need to change such default. But if you want to change it
```
sudo vim /etc/nginx/nginx.conf
```
Now if we scrool down to http
one can set any variable for example client_max_body_size 5M;
Now after saving them one need to restart nginx
```
sudo systemctl restart nginx
```
Now it should work
Actually we have completely functional website
However we have only ip address. We can also buy a domain name
also how to add ssl certificate to have a https domain name
Now we can completely delete the server.
go to linode. and go the application server. right settings menu and goto settings. At the bottom there is a button, which says delete linode

Buy domain name

Need to register to domain name.
Domain registers – corey using namechip
First register and then search for domain name
- Normally difficult to find if available then will show the name and price. Also avialble other thing with same name (.com, .net etc.)
- There are some additional add-on with extra cost, one actually can also add them. However corey donot use that.
- There is a free addon called whoisguard. Corey recommended to use that because it protects privacy.
After paying for domain name, there is button called manage. Now we are ready to connect to our applciation.
Goto server Normally for this blog it was Linode. In linode there is option called Domains. In the right side there is some documentation Linode Docs DNS Manager. This will be used to connect the domain name with the server.
In the documentation
- buy domain
- go to domain page and manage there is a opiton called NAMESERVERS and select custom DNS and change the name from linode server. Normally nsl.linode.com and so on..
- There was right click sign which will save the name
- We need to wait some time. 24 hours to 48 hours.
- During this waiting time we can do something in linode server in the meantime.
- Add Domain zone and adding some basic DNS recor ds
- In linode server go to domain page and right side there is a plus sign which will create a form. Domain –> myawesomeapp.com and there will be email address admin@myawesomeapp.com. Add some tags, can be empty. Click on create
- There will be different records. We will add a AAAA Record. We need a hostname and ip address. Hostname will be www and ip will be ip address of our application. Now save them the other option was like it was.
- Now do reverse DNS
- go to linode and go the application and select Networking and IPv4 right side there will be some dotted sign and if we click on that we will see the two options, View and Edit RDNS. we need to click on Edit DNS. There was something. We need to put our domain name, in our case www.myawesomeapp.com. save that, if there is error, may be we need to wait some time
- Now copy the domain name and put them in the server.
- Test all the things in the app
- if we remove www. from the app it is not working. If we need to work it without www. we need to go again in Linode. go to myawesomeapp.domain.
- Add another _AAAA record. In the hostname set the hostname blank and ip address will be our app ip address. It also take some time. may be 10-30 minutes and in worst case one day
# Enable HTTPS with a free SSL certificate/ TLS Certificate
- Right now in our wegsite it is written it is not secure. We will user Let’s Encrypt.
- In Let’s Encrypt website, there is link for differnt operating system. For shell script there is a link for certbot. We are using nginx and ubuntu. We need to select those options
- Then there will be documentation for that.
- Available code for Nginx and Ubuntu
```
  sudo apt-get update
  sudo apt-get install software-properties-common
  sudo apt-get-repository universe
  sudo apt-get-repository ppa:certbot/certbot
  sudo apt-get update
  sudo apt-get install python-certbot-nginx
```
- Now we need to change nginx configuration. Now server name is ipaddress, now we need to change to our domain name
```
  sudo vim /etc/nginx/sites-enabled/flaskblog 
```
  In server { listen 80; server_name www.awesomeapp.com; }
- Now save the file and run the commmand

    sudo certbot --nginx
    sudo certbot renew --dry-run

after first command ask some question. Number 2 Ridirect to secure site.

It will show some changes are done on configuration file let us see what has it done ```bash sudo vim /etc/nginx/sites-enabled/flaskblog

one can see that it will redirect to secure opiton of the file

* If we want to see __nginx__ we will see there is some issue. 
``` bash
nginx -t

here t means test the configuration
Now if we use same command usind sudo, then there will be no issue
```
sudo nginx -t
```
Allow https trafic to firewall
```
sudo ufw allow https
```
Now restart the server
```
sudo systemctl restart nginx
```
Automate certificate after sometime
We will dryrun. Dryrun actually to test everything is working as expercted or not, it will simulate everything but don’t do anything
```
sudo certbot renew --dry-run
```
So we will do cron-job
```
sudo crontab -e
```
Now we need to choose an editor. we will select 2 for vim
```
 30 4 1 * * sudo certbot renew --quiet 
```
quiet just do quietly Now save the file # Crontab example and tutorial
Here 30[Minutes] 4[Hours] 1[Day] *[Month:every month] *[Day of week:every day of the week] 0–> sunday 6 –> saturday comman for multiple values. 0,6 means only sunday and saturday */10 * * * *–> means we want something to run every 10 minutes
for example if we want something to run every 10 minutes only from midinight to 5 a.m 0 0-5 * * *

if we want something to run every 30 Minutes in common business hours */30 9-17 * * 1-5 echo 'hello' >> /tmp/test.text

For a specific user crontab -u user2 -e command
For a root sudo crontab -l–> list all crontab for root

now with e command one can edit the command crontab -r remove all crontab

cronGuru is a website where you can test those cronjob

tags:

Hasan's Post

Tutorial repository

Deploying app in a linux server

Using Linode Deploying on Linux Server

Entering first on the Server

Adding new user

Deploy a firewall

Get Application to the server

Creating requirement.txt

Copy project to server

Create Virtual Environment on server

Create Configuration file with all of our environment varaiable

Run the flask app inside the server

Run the application with nginx and gunicorn

nginx and gunicorn configuration

Monitor gunicorn

Buy domain name

Allow https trafic to firewall

Automate certificate after sometime

Deploying app in a linux server

Using Linode Deploying on Linux Server

Entering first on the Server

Adding new user

Logout from the server and login as this new user

SSH keybased authentication. Login without password

Disallow root login over ssh

Deploy a firewall

Get Application to the server

Creating requirement.txt

Copy project to server

Create Virtual Environment on server

Create Configuration file with all of our environment varaiable

Run the flask app inside the server

Run the application with nginx and gunicorn

nginx and gunicorn configuration

Monitor gunicorn

Buy domain name

Allow https trafic to firewall

Automate certificate after sometime