Tuesday, April 21, 2015

This blog has been moved to http://oopsmonk.github.io
Posted by at 1 comment:

Friday, December 12, 2014

Using Git With Multiple SSH Keys and Accounts

Generating SSH keys for GitHub

Here are github account and work account.

  • GitHub:
    SSH Key: github_id_rsa
    Account: oopsmonk

  • Work:
    SSH Key: work_id_rsa
    Account: SamChen

Add SSH config File

Modify ~/.ssh/config

# Default github
Host github.com
    HostName github.com
    PreferredAuthentications publickey
    IdentityFile ~/.ssh/github_id_rsa

# Work git server
Host work.gitserver.com
    HostName work.gitserver.com
    PreferredAuthentications publickey
    IdentityFile ~/.ssh/work_id_rsa

Git Repository Configuation

  • GitHub Project:

    $ git clone https://github.com/abc/projectA.git  
$ cd projectA
#github account
$ git config user.name "oopsmonk"
$ git config user.email "oopsmonk@example.com.tw"

  • Working Project:

    $ git clone https://work.com.tw/repo/projectW.git  
$ cd projectW
#working account
$ git config user.name "SamChen"
$ git config user.email "SamChen@example.com.tw"

Now you can deal with git repositories using different accounts.

(Additional) Using ssh-agent to access GitHub without password

Check key list

# No ssh-agent running 
$ ssh-add -l
Could not open a connection to your authentication agent.
$

Enable ssh-agent (Ubuntu12.04):

#enable ssh-agent
$ eval `ssh-agent -s`

#add keys to agent
$ cd ~/.ssh
$ ssh-add work_id_rsa github_id_rsa
Identity added: work_id_rsa (work_id_rsa)
Identity added: github_id_rsa (github_id_rsa)

#list added keys
$ ssh-add -L

#delete keys in agent  
$ ssh-add -D

Now you can commit modifications to git repository without password.

Reference:
Understanding ssh-agent and ssh-add
How can I run ssh-add automatically, without password prompt?

Posted by at No comments:
Labels: ,

Wednesday, December 10, 2014

pyenv Quick Start (Utunbu 14.04)

installation

Requirements

$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm

install pyenv

$ cd ~
$ git clone git://github.com/yyuu/pyenv.git .pyenv
$ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
$ echo 'eval "$(pyenv init -)"' >> ~/.bashrc

install pyenv-virtualenv

$ git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv 
$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc
$ exec $SHELL

Common Usage (Command Reference)

List all available versions

$ pyenv version -l

Install python2.7.8 to pyenv

$ pyenv install 2.7.8

List installed versions

$ pyenv versions
* system (set by /home/oopsmonk/.pyenv/version)
  2.7.8
  3.4.2

Creating a virtualenv

$ pyenv virtualenv 2.7.8 mypy-2.7.8

List virtualenvs

$ pyenv virtualenvs
  mypy-2.7.8 (created from /home/oopsmonk/.pyenv/versions/2.7.8)
  mypy-3.4.2 (created from /home/oopsmonk/.pyenv/versions/3.4.2)

# current versions 
$ pyenv versions
  * system (set by /home/oopsmonk/.pyenv/version)
  2.7.8
  3.4.2
  mypy-2.7.8
  mypy-3.4.2

Use python via virtualenv

# show current version
oopsmonk@VBox:~/markdown-note$ python --version
Python 2.7.6

# Change to other version
oopsmonk@VBox:~/markdown-note$ pyenv activate mypy-2.7.8
(mypy-2.7.8)oopsmonk@VBox:~/markdown-note$ python --version
Python 2.7.8

# Deactivate
(mypy-2.7.8)oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv deactivate
oopsmonk@oopsmonk-VBox:~/markdown-note$  python --version
Python 2.7.8
# Why current version is 2.7.8? it's supposed to 2.7.6.

oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv activate mypy-3.4.2
(mypy-3.4.2) oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv deactivate
oopsmonk@oopsmonk-VBox:~/markdown-note$ python --version
Python 3.4.2

oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv versions
  system
  2.7.8
  3.4.2
  mypy-2.7.8
* mypy-3.4.2 (set by PYENV_VERSION environment variable)

# delete a virtualenv   
$ pyenv uninstall mypy-2.7.8

# Go back to original system version  
$ alias pyenv_deactivate='pyenv deactivate && unset PYENV_VERSION'
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv activate mypy-3.4.2
(mypy-3.4.2) oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv_deactivate
oopsmonk@oopsmonk-VBox:~/markdown-note$ python --version
Python 2.7.6

Use python via pyenv

# Global python version
$ pyenv global 

# python version in current folder
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv local mypy-3.4.2
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv version
mypy-3.4.2 (set by /home/oopsmonk/markdown-note/.python-version)
oopsmonk@oopsmonk-VBox:~/markdown-note$ cd ..
oopsmonk@oopsmonk-VBox:~$ pyenv version
system (set by /home/oopsmonk/.pyenv/version)
oopsmonk@oopsmonk-VBox:~$ cd -
/home/oopsmonk/markdown-note
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv version
mypy-3.4.2 (set by /home/oopsmonk/markdown-note/.python-version)
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv local --unset
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv version
system (set by /home/oopsmonk/.pyenv/version)

# python version in current shell 
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv shell mypy-3.4.2
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv version
mypy-3.4.2 (set by PYENV_VERSION environment variable)
oopsmonk@oopsmonk-VBox:~/markdown-note$ cd -
/home/oopsmonk
oopsmonk@oopsmonk-VBox:~$ pyenv version
mypy-3.4.2 (set by PYENV_VERSION environment variable)
oopsmonk@oopsmonk-VBox:~$ pyenv shell --unset
oopsmonk@oopsmonk-VBox:~$ pyenv version
system (set by /home/oopsmonk/.pyenv/version)
oopsmonk@oopsmonk-VBox:~$ cd -
/home/oopsmonk/markdown-note
oopsmonk@oopsmonk-VBox:~/markdown-note$ pyenv version
system (set by /home/oopsmonk/.pyenv/version)
Posted by at 385 comments:
Labels:

Monday, July 21, 2014

Raspberry Pi Monitor

Use RRDTool monitor Raspberry Pi, include CPU temperture, Memory usage, Disk I/O, Network I/O...

Install

install packages

sudo apt-get install libcairo2-dev libpango1.0-dev libglib2.0-dev libxml2-dev librrd-dev python2.7-dev rrdtool python-rrdtool
wget https://pypi.python.org/packages/source/p/psutil/psutil-2.1.1.tar.gz
tar xf psutil-2.1.1.tar.gz
cd psutil-2.1.1
sudo python setup.py install

Download or clone rpi-monitor on github
https://github.com/oopsmonk/rpi-monitor

Setup Crontab

By defualt, the cron.log is disabled in Raspbian. To enable it:

sudo vi /etc/rsyslog.conf

find the line and uncomment it.

# cron.*                          /var/log/cron.log

Restart rsyslog via:

sudo /etc/init.d/rsyslog restart

Modify crontab

crontab -e

Add schedule as below

#data collection every 5 minutes
*/5 * * * * /path/to/rpi-monitor/rpi_monitor.py
#generate daily graph report at 00:01
1 0 * * * /path/to/rpi-monitor/graphReport.py -1d
#generate weekly graph report at 00:03 on Monday
3 0 * * 1 /path/to/rpi-monitor/graphReport.py -1w

Report example

Raspberry Pi Hardware

CPU Temperture

The temperture drop to 44 because I add a fan on CPU.

CPU Used Percentage

PID Count

Memory Usage

Mount point Usage

Mount Point Percentage

HDD I/O

eth0 I/O

eth1 I/O

Posted by at 2 comments:
Labels:

Wednesday, August 7, 2013

Web dev example : JSON & jQuery Mobile & Bottle.

Bottle is a fast, simple and lightweight WSGI micro web-framework for Python.
jQuery Mobile base on jQuery for mobile device.
jQuery vs. jQuery Mobile vs. jQuery UI

Install bottle:

$ sudo apt-get install python-setuptools
$ easy_install bottle

Demo server deployment

file structure:

BottlejQuery
├── bottleJQuery.py
└── index.html

run command:

$ ./bottleJQuery.py

connect to server:

http://localhost:8080/bottle

Building simple web server use bottle

bottleJQuery.py

#!/usr/bin/env python
from bottle import route, static_file, debug, run, get, redirect
from bottle import post, request
import os, inspect, json

#enable bottle debug
debug(True)

# WebApp route path
routePath = '/bottle'
# get directory of WebApp (bottleJQuery.py's dir)
rootPath = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))

@route(routePath)
def rootHome():
    return redirect(routePath+'/index.html')

@route(routePath + '/<filename:re:.*\.html>')
def html_file(filename):
    return static_file(filename, root=rootPath)

@get(routePath + '/jsontest')
def testJsonGET():
    print "GET Header : \n %s" % dict(request.headers) #for debug header
    return {"id":2,"name":"Jone"}

@post(routePath + '/jsontest')
def testJsonPost():
    print "POST Header : \n %s" % dict(request.headers) #for debug header
    data = request.json
    print "data : %s" % data 
    if data == None:
        return json.dumps({'Status':"Failed!"})
    else:
        return json.dumps({'Status':"Success!"})

run(host='localhost', port=8080, reloader=True)

Test GET request:

$ curl -i -X GET http://localhost:8080/bottle/jsontest

HTTP/1.0 200 OK
Date: Wed, 07 Aug 2013 16:03:53 GMT
Server: WSGIServer/0.1 Python/2.7.3
Content-Length: 25
Content-Type: application/json

{"id": 2, "name": "Jone"}

bottle debug message:

GET Header :
 {'Host': 'localhost:8080', 'Content-Type': 'text/plain', 'Content-Length': '', 'Accept': '*/*', 'User-Agent': 'curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3'}
localhost - - [08/Aug/2013 00:03:53] "GET /bottle/jsontest HTTP/1.1" 200 25

Test POST request:

$ curl -X POST -H "Content-Type: application/json" -d '{"name":"OopsMonk","pwd":"abc"}' http://localhost:8080/bottle/jsontest
{"Status": "Success!"}

bottle debug message:

POST Header :
 {'Host': 'localhost:8080', 'Content-Type': 'application/json', 'Content-Length': '31', 'Accept': '*/*', 'User-Agent': 'curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3'}
data : {u'pwd': u'abc', u'name': u'OopsMonk'}
localhost - - [08/Aug/2013 00:04:56] "POST /bottle/jsontest HTTP/1.1" 200 22

Building simple web page

Create 4 buttons in the index.html file:

  • jQuery.getJSON : use jQuery.getJSON API.
    bottle debug message:

    GET Header :
 {'Content-Length': '', 'Accept-Language': 'zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip, deflate', 'Host': '127.0.0.1:8080', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0', 'Connection': 'close', 'X-Requested-With': 'XMLHttpRequest', 'Referer': 'http://192.168.1.38/bottle/index.html', 'Content-Type': 'text/plain'}  
localhost - - [08/Aug/2013 00:42:08] "GET /bottle/jsontest HTTP/1.0" 200 25

  • jQuery.post : use jQuery.post API.
    Unfortunately it's not work, because the content-Type is fixed 'application/x-www-form-urlencoded; charset=UTF-8', that's a problem when using bottle.request.json API to retrieve JSON from request.
    For this perpose we have to overwirte jQuery.post or use jQuery.ajax.
    bottle debug message:

    POST Header :
 {'Content-Length': '22', 'Accept-Language': 'zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip, deflate', 'Host': '127.0.0.1:8080', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0', 'Connection': 'close', 'X-Requested-With': 'XMLHttpRequest', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache', 'Referer': 'http://192.168.1.38/bottle/index.html', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'}
data : None
localhost - - [08/Aug/2013 00:55:17] "POST /bottle/jsontest HTTP/1.0" 200 21

  • jQuery.ajax GET, jQuery.ajax POST : use jQuery.ajax API.
    GET debug message:

    GET Header :
 {'Content-Length': '', 'Accept-Language': 'zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip, deflate', 'Host': '127.0.0.1:8080', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0', 'Connection': 'close', 'X-Requested-With': 'XMLHttpRequest', 'Referer': 'http://192.168.1.38/bottle/index.html', 'Content-Type': 'text/plain'}
localhost - - [08/Aug/2013 01:00:17] "GET /bottle/jsontest HTTP/1.0" 200 25

    POST debug message:

    POST Header :
 {'Content-Length': '22', 'Accept-Language': 'zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip, deflate', 'Host': '127.0.0.1:8080', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0', 'Connection': 'close', 'X-Requested-With': 'XMLHttpRequest', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache', 'Referer': 'http://192.168.1.38/bottle/index.html', 'Content-Type': 'application/json; charset=utf-8'}
data : {u'id': 3, u'name': u'Ping'}
localhost - - [08/Aug/2013 01:01:40] "POST /bottle/jsontest HTTP/1.0" 200 22

index.html

<!DOCTYPE html>
<html>
<head>
<title>Bottle & jQuery Mobile</title>

<link rel="stylesheet" href="http://code.jquery.com/mobile/1.3.2/jquery.mobile-1.3.2.min.css" />
<script src="http://code.jquery.com/jquery-1.9.1.min.js"></script>
<script src="http://code.jquery.com/mobile/1.3.2/jquery.mobile-1.3.2.min.js"></script>

</head>

<body>
<h3>This is a example using jQuery Mobile and pyhton bottle fromwork. </h3>
<div data-role="controlgroup" data-type="horizontal">
    <a href="#" id="btnGetJSON" data-role="button">jQuery.getJSON</a>
    <a href="#" id="btnPOSTJSON" data-role="button">jQuery.post</a>
    <a href="#" id="btnAJAXGet" data-role="button">jQuery.ajax GET</a>
    <a href="#" id="btnAJAXPOST" data-role="button">jQuery.ajax POST</a>
</div>

<script>
    //button action 
    $("#btnGetJSON").click(function(){
        $.getJSON("jsontest", function(data){
            $.each(data, function(index, value){
                alert("index: " + index + " , value: "+ value);
            });
        });
    });

    $("#btnPOSTJSON").click(function(){
        alert("Not easy work on bottle !!!");
        /*
        It's not work, send JSON via jQuery.post().
        because the content-Type is not 'application/json',
        The content type is fixed :
        'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'
        that's a problem on bottle.request.json.
        */

        $.post(
        "jsontest", 
        JSON.stringify({"id":3, "name":"Ping"}), 
        function(ret_data, st){
            $.each(ret_data, function(index, value){
                alert("index: " + index + " , value: "+ value);
            });
            alert("Server return status : " + st); 
        },
        'json'
        );
    });

    $("#btnAJAXGet").click(function(){
        $.ajax
        ({
            url: 'jsontest',
            success: function(data){
                $.each(data, function(index, value){
                    alert("index: " + index + " , value: "+ value);
                });
            },
            /*
            if dataType not set, the Accept in request header is:
            'Accept': '* / *'
            dataType = json :
            'Accept': 'application/json, text/javascript, * /*; q=0.01'
            */
            dataType: 'json'
        });
    });

    $("#btnAJAXPOST").click(function(){
        var post_data = {"id":3, "name":"Ping"};
        $.ajax
        ({
            type: 'POST',
            url: 'jsontest',
            data:JSON.stringify(post_data),
            contentType: "application/json; charset=utf-8",
            dataType: 'json',
            success: function(data){
                $.each(data, function(index, value){
                    alert("index: " + index + " , value: "+ value);
                });
            }
        });
    });
</script>
</body>
</html>
Posted by at 1 comment:
Labels: , ,

Wednesday, July 31, 2013

Sending HTML Mail Using SMTP With Authorization

Here is a text/plain MIME type parts in official exmaple code.
I remove it from my sample code, because it's not show up in mail at Office Outlook 2010.

#!/usr/bin/env python
"""
File name: sendMail.py
Python send HTML mail using SMTP with authorization

Usage :
./sendMail.py to@gmail.com Subtitle [ FilePath | txt ]
"""

import smtplib
import sys,traceback
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from time import gmtime, strftime

#log file location
log_path = "./mail-log";
#log timestamp format
logger_tstamp = "%Y-%m-%d %H:%M:%S"

#SMPT server
smtp_server = "smtp.gmail.com"
#gmail 465 or 578
smtp_port = 587

#mail from 
user = "from@gmail.com"
pwd = "password"

# log method
class Logger:
    file = 1
    err = 2

class ContentType:
    file = 1
    txt = 2

#select log method 
debug_log = Logger.err
#select content type
content_type = ContentType.txt

def debug(msg):
    if debug_log == Logger.file:
        with open(log_path, 'a') as log_file:
            log_file.write(strftime(logger_tstamp, gmtime()) + msg)
        log_file.close()

    elif debug_log == Logger.err:
        sys.stderr.write(strftime(logger_tstamp, gmtime()) + msg)

    else: 
        print(strftime(logger_tstamp, gmtime()) + msg)

#check argument number
arg_count = len(sys.argv)
if arg_count !=4:
    debug("[Error]: invalid argument\n")
    sys.exit(1)
try:
    mail_to = sys.argv[1]
    subject = sys.argv[2]
    if content_type == ContentType.file:
        content_path = sys.argv[3]
        with open(content_path, 'r') as f_content:
            content = f_content.read()
        f_content.close()
    else:
        content = sys.argv[3]

    serv = smtplib.SMTP(smtp_server, smtp_port)
#    serv.ehlo()
    serv.starttls()
#    serv.ehlo()
    serv.login(user, pwd);
    msg = MIMEMultipart('alternative')
    msg['Subject'] = subject
    msg['From'] = user
    msg['To'] = mail_to
    part1 = MIMEText(content, 'html')
    msg.attach(part1)
    serv.sendmail(user, mail_to, msg.as_string())
    serv.quit()
    debug("[Debug] subject : " + subject + "\n")
except:
    debug("[Error]: send mail error\n")
    debug("[Error]:\n" + traceback.format_exc())
    sys.exit(2);

Reference:
email: Examples
smtplib
How to send email in Python via SMTPLIB

Posted by at No comments:
Labels:

Friday, July 5, 2013

Redirect and Save iptables on Ubuntu 12.04

Redirect port 8080 to 80

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

Check iptables setting

sudo iptables -t nat -L

Save configure to iptables.rules

sudo iptables-save > /etc/iptables.rules

Save Solution #1

Configre /etc/network/interfaces

iface eth0 inet dhcp  
    pre-up iptables-restore < /etc/iptables.rules

Save Solution #2

Configure /etc/network/if-pre-up.d/iptablesload

#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

Configure /etc/network/if-post-down.d/iptablessave

#!/bin/sh
iptables-save -c > /etc/iptables.rules
if [ -f /etc/iptables.downrules ]; then
    iptables-restore < /etc/iptables.downrules
fi
exit 0

Change permissions

sudo chmod +x /etc/network/if-post-down.d/iptablessave  
sudo chmod +x /etc/network/if-pre-up.d/iptablesload

Reference:
Tomcat: redirecting traffic from port 8080 to 80 using iptables
IptablesHowTo

Posted by at No comments:
Labels: ,
Subscribe to: Posts (Atom)