❌

普通视图

发现新文章，点击刷新页面。
昨天以前herrkaefer

Python dev workflow on macOS

✇herrkaefer
作者 herrkaefer
2018年10月25日 04:57

Here is the new workflow in place of the old popular one using virtualenv, pip, requirements.txt and virtualenvwrapper.

Tools

  • pyenv for Python version installation and switch
  • Pipenv for virtualenv creation and package management for each project
  • Pipenv-Pipes for Pipenv environment switch

Pipenv is the central tool. pyenv provides specific version of Python (and pip) for Pipenv to initialize its virtualenv. Pipes is just a handy tool for fast navigation between Pipenv’s virtualenvs (like virtualenvwrapper’s workon), which may be built into Pipenv in the future.

Basic workflow

pyenv

To install:

$ brew install pyenv

then add eval "$(pyenv init -)" to shell (e.g. ~/.zshrc)

To show available Python versions to install:

$ pyenv install --list

To install a Python version:

$ pyenv install <version>

Pipenv

Install Pipenv:

$ brew install pipenv

To install environment with specific Python version other than system default, this version must exist in system, so we can use pyenv to install the version first, then switch to its context.

$ pyenv shell <version>
$ pipenv --python <version>

After this, pyenv will be of no use because corresponding Python and pip have been installed into Pipenv’s context, unless you update to a new Python version.

Tips

Create Pipfile from requirements.txt

To use Pipenv for an old project with requirements.txt available, we can first

$ pipenv install

Pipfile will be automatically created from requirements.txt, with all packages installed with specific version numbers.

Second, run

$ pipenv graph

to show a dependency tree graph, from which we can identify less packages that we want. Then we can manually edit Pipfile to keep only these key packages. Note that it does not need to make a Pipfile without any redundant (i.e. all packages are leaf nodes in the dependency tree).

Last, run pipenv install again.

Time saving with --skip-lock

Uninstall packages that are not in Pipfile

Manually deleted packages in Pipfile would not be automatically uninstalled, we can use

$ pipenv clean

Sublime Text syntax highlighting of Pipfile

(Tested for ST 3)

  1. Install package TOML for TOML syntax highlighting. (Pipfile uses TOML format.)
  2. Install package ApplySyntax for custom syntax hightlighting for non-default format
  3. Add the following to ApplySyntax settings:
"syntaxes": [
    {
        "syntax": "TOML/TOML",
        "rules": [
            {"file_path": ".*\\Pipfile$"}
        ]
    },
    {
        "syntax": "JavaScript/JSON",
        "extensions": ["Pipfile.lock"]
    },
]

The post Python dev workflow on macOS appeared first on herrkaefer.

Handle annoying operations of objects in Realm DB

✇herrkaefer
作者 herrkaefer
2017年6月13日 04:57

Problem

Update value of object in Realm DB is annoying because the task should be within a writing transition.

Solution

Extend the Realm’s Object class to provide convient methods to do these things:

  • update: to set/update properties’ values.
  • remove: to remove object from Realm
// RealmHelper.swift
import RealmSwift

extension Object {
    // Update property value in Realm
    func update(_ property: String, value: Any?) {
        let realm = try! Realm()
        try! realm.write {
            self.setValue(value, forKey: property)
        }
    }
    
    // Remove object from Realm
    func remove() {
        let realm = try! Realm()
        try! realm.write {
            realm.delete(self)
        }
    }
}

The post Handle annoying operations of objects in Realm DB appeared first on herrkaefer.

Move Jekyll blog to Ubuntu VPS

✇herrkaefer
作者 herrkaefer
2017年2月21日 05:57

(Tested on: Linode with Ubuntu 16.04 LTS)

Install Jekyll on Ubuntu VPS

sudo apt update && apt -y upgrade
sudo apt install ruby ruby-dev ruby-bundler zlib1g-dev build-essential make gcc git jekyll bundler

Automated deployment with Git

On VPS,

mkdir ~/blog.git # the site source
mkdir -p ~/www/blog # the built site

Setup git hook to automatically build the site whenever after site source is updated:

cd ~/blog.git
git init --bare
cp hooks/post-update.sample hooks/post-receive

--bare means that our folder will have no source files, just the version control.

Edit hooks/post-receive:

#!/bin/sh
GIT_REPO=$HOME/blog.git
TMP_GIT_CLONE=$HOME/tmp/blog
PUBLIC_WWW=$HOME/www/blog

git clone $GIT_REPO $TMP_GIT_CLONE
jekyll build -s $TMP_GIT_CLONE -d $PUBLIC_WWW
rm -Rf $TMP_GIT_CLONE
exit

Serve static site with Nginx

Install nginx:

sudo apt install nginx

Add a configuration:

sudo nano /etc/nginx/conf.d/blog.conf

with content:

server {
  # listen on http (port 80)
  # remove the "default_server" if you are running multiple sites off the same VPS
  listen 80 default_server;

  # the IP address of your VPS
  server_name herrkaefer.com;
  # see http://nginx.org/en/docs/http/server_names.html for options
  # to use your own domain, point a DNS A record at this IP address
  # and set the server name to (eg.) "blog.example.com"

  # the path you deployed to. this should match whatever was in your
  # Capistrano deploy file, with "/current" appended to the end
  # (Capistrano symlinks to this to your current site)
  root /home/herrkaefer/www/blog;
  index index.html

  # how long should static files be cached for, see http://nginx.org/en/docs/http/ngx_http_headers_module.html for options.
  expires 1d;
}

Then remove default config file and restart Nginx:

sudo rm /etc/nginx/sites-enabled/default
sudo systemctl restart nginx

Remember to allow Nginx port if the firewall is enabled.

Local machine setup

On local machine, in Jekyll repo folder, add remote repository:

cd [jekyll_blog_folder]

Show remotes:

git remote -v

Add remote named web:

git remote add web username@[host]:~/blog.git

If you need to specify other SSH port, you should do:

git remote add web ssh://username@[host]:[port][absolutePathTo]/blog.git

note that ssh:// can not be omitted.

Write and publish

After writing,

git commit -am "blog updated"
git push web master

Troubleshoot

Nginx 403 Forbidden

nginx 403 Forbidden 排错记录 – 简书

The post Move Jekyll blog to Ubuntu VPS appeared first on herrkaefer.

Deploy Python web application on Ubuntu server

✇herrkaefer
作者 herrkaefer
2016年10月9日 04:57

Components used for this article:

Suppose that we already installed an Ubuntu image on a Linode plan.

Update packages

sudo apt-get update && apt-get upgrade

DNS resolution

Add a A record for the domain name of the application.

Nginx

Install Nginx

sudo apt-get install nginx

Configure Ngnix

Create a cofiguration file under /etc/nginx/conf.d/ with name [app_name].conf.

sudo nano /etc/nginx/conf.d/[app_name].conf

Configure without https

Template of /etc/nginx/conf.d/[app_name].conf:

upstream app_server_wsgiapp {
    server localhost:8000 fail_timeout=0;
}

server {
    listen 80;
    # make sure to change the next line to your own domain name!
    server_name appname.example.com;
    access_log /var/log/nginx/appname.access.log;
    error_log /var/log/nginx/appname.error.log info;
    keepalive_timeout 5;

    # nginx serve up static files and never send to the WSGI server
    location /static {
        autoindex on;
        alias /home/username/appname/static;
    }

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            if (!-f $request_filename) {
                proxy_pass http://app_server_wsgiapp;
                break;
            }
    }

    # this section allows Nginx to reverse proxy for websockets
    location /socket.io {
        proxy_pass http://app_server_wsgiapp/socket.io;
        proxy_redirect off;
        proxy_buffering off;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

Configure with https

(to add)

Restart nginx

Delete default website file:

sudo rm /etc/nginx/sites-enabled/default

Apply new configuration:

sudo service nginx restart

PostgreSQL

Install PostgreSQL

sudo apt-get install postgresql libpq-dev postgresql-client-common postgresql-client

then create database and new user

# switch to postgres user
sudo su - postgres

# create database
createdb dbname

# create a non-root database user and set password
createuser --superuser username
psql
ALTER USER username WITH PASSWORD 'password'


(For me, it seems that the last line would take effect, but we can change the password later.)

Press CTRL-D to quit psql, and CTRL-D again to quit postgres user.

Now database dbname is created and user username is able to access the database.

# Connect to database
psql dbname

# Show tables
dt

Change password of current user:

password

Redis

Install Redis

sudo apt-get install redis-server

Test using redis-cli.

Python application

Install pip, virtualenv

sudo apt-get install python3-pip
sudo apt-get install python3.4-venv
sudo python3 -m pip install virtualenv virtualenvwrapper

Note: python3.4-venv not python3-venv

Make virtualenv

# set an environment variable to where you want your virtual environment
export VENV=~/env
# create the virtual environment
python3 -m venv $VENV

Upgrade packages in virtualenv

$VENV/bin/pip install --upgrade pip setuptools

“Why use $VENV/bin/pip instead of source bin/activate, then pip?” From Pyramid docs:

$VENV/bin/pip clearly specifies that pip is run from within the virtual environment and not at the system level.

activate drops turds into the user’s shell environment, leaving them vulnerable to executing commands in the wrong context. deactivate might not correctly restore previous shell environment variables.

Although using source bin/activate, then pip, requires fewer key strokes to issue commands once invoked, there are other things to consider. Michael F. Lamb (datagrok) presents a summary in Virtualenv’s bin/activate is Doing It Wrong.

Ultimately we prefer to keep things clear and simple, so we use $VENV/bin/pip.

Pyramid

Install Pyramid

# install pyramid
$VENV/bin/pip install pyramid
# or for a specific released version
$VENV/bin/pip install "pyramid==1.7.3"

Demo app

We create a demo app using scaffold provided by Pyramid.

$VENV/bin/pcreate -s starter demo
cd demo
$VENV/bin/pip install -e .

WSGI server

Install Gunicorn

$VENV/bin/pip install gunicorn

Run the demo app:

cd ~/demo
$VENV/bin/gunicorn --paste development.ini -b :8000

8000 is the port gunicorn listens from the nginx reverse proxy, which we set in /etc/nginx/conf.d/demo.conf.

Open a browser and visit the URL and the app should be running.

Start gunicorn server with supervisor

Install supervisor

sudo apt-get install supervisor

Configure supervisor. Create a file /etc/supervisor/conf.d/[app_name].conf, and add lines: (e.g., for demo app, the file name should be demo.conf)

[program:demo] 
environment=DEBUG=False
command=/home/[username]/venv/bin/gunicorn --paste [path/to/app/]development.ini -b :8000 --chdir [path/to/app]
directory=/home/[username]/demo 
user=[username]
autostart=true 
autorestart=true
redirect_stderr=True

Start supervisor service:

sudo service supervisor start

Visit the URL and the app should be running.

The post Deploy Python web application on Ubuntu server appeared first on herrkaefer.

❌
❌