Random Bits

Technical Log from Didik Setiawan

Workflow on Contributing to Open Source Project

In the open source world, different project comes with different workflow, using different medium. For instance, Linux Kernel development, use mailing list to gather patches for many developer around the world. The maintainer pick patches from developer with careful supervision. Then, they maintain release of the stable version, while Linus Torvalds himself maintain the mainline stream. Other project, Git – a revision control system. It use same model as Linux Kernel, patches and conversations go to mailing list. With additional part, for the Continuous Integration system, it is recommended to use Github and Travis CI. Although they will reject all pull request from there. Another example, Wget2 use Github mainly to perform collaboration (But currently, they start to migrate to Gitlab[1]). Issues are discussed here. Pull request also getting merged here. Although, it also have mailing list to discuss issues and problems. Here I share my experience how I contribute to open source, so my code could be merged in the project upstream. For an example, I will write several steps of my way doing Wget2 project for my GSoC 2017 application. Below are some points I follow:

Obtain Source Code

Startup procedures:

Through Github interface, I fork (copy) the project, then I download it locally to my computer.

git clone https://github.com/dstw/wget2
git remote add upstream git://git.savannah.gnu.org/wget/wget2.git
git pull --all

I create my own branch for each “task” I am working on and make my commits within it. When done, I sync with upstream, rebase/merge and make pull/merge request.

I should prevent to make changes to my branch “master”, because this should always reflect the original upstream repo. This makes it easy for me to update my “master” branch with changes from the “original/upstream” master branch.

Sync my Github repo with upstream:

git checkout master
git pull --all
git merge --ff-only upstream/master
git push #pushing the new commits from upstream/master to your github master branch)

Sync my ‘new-feature’ branch before generating patches:

git checkout new-feature
git rebase master
git push -f

The last command will push updated tree to my github new-feature branch. Assume ‘new-feature’ is a private branch where I can do all the dirty things that I shouldn’t on public/shared branches. Then, as soon as my patch has been accepted, I will remove the branch ‘new-feature’ locally and from Github.

Build the Executables

As prerequisite, install all mandatory software as listed in README.

Find the Problems

I usually look at Wget2 Github pages to find issues and problems which appropriate to newcomer like me.

Make Changes to the Code

In this part, my knowledge in coding skill are truly challenged. Wget2 use C as its programming language. I just add some test based on clues from the project maintainer.

Create Pull/Merge Request

After I feel my code ready to merge, I push my commit to upstream and make a pull request from Github interface.

Wait for Review

After create a merge request, I wait for feedback, criticism, suggestions, etc. from other developer. I try to respond to it. Follow up with improved versions of my change. Even for a trivial patch I shouldn’t be surprised if it takes two or more iterations before my patch is accepted/merged. This is the best part of participating in the community; it is becomes my chance to get personalized instruction from very experienced peers.

Propose New Change

After getting a review, do some improvement. After all, make a new commit. Here, there is option to overwrite my existing commits or create new merge commits to keep history.

git push origin new-feature

If there is a conflict, Git will ask me to resolve the problem this. Then, try to push and merge commits. This won’t remove any existing commits. And if I don’t want to keep any commits history, I usually force to push:

git push -f origin new-feature

This is considered harmful. Before I do this, I will make sure that I know what I do. Also, forcing commits is should only do if it believed to final commits or at least near final, I should not do any more change to it.

Finally, after some iteration, if project maintainer agree with my changes, then they will merge it into their repository. If not, still I could learn a lot from the process.

[0] http://lists.gnu.org/archive/html/bug-wget/2017-04/msg00052.html

Preparing GSoC

GSoC (Google Summer of Code), a program where university students spend their 3 month summer break coding on an open source project.

That time finally comes. After almost one year I have been lurking, observing, it’s time to decide whether I will participate or not. It’s not easy to find organization who fit with who I am. But anyway, this year, I will try.
Despite administrative prerequisite, such as proof of enrollment, there are some technical stuff involving coding activities we need to do, which is fun. This is a list of what I’ve done so far in order to be considered as good candidate:

Pick a project with programming language that I passionate about

There is a word “code” in Google Summer of Code, meaning programming will be my main activity in this program. So, I must find which project that has programming language, that I familiar with. This is to avoid, I will stuck about the tasks that will assigned to me as a program applicant.
Actually, it can be done even long ago before the program being announced. I can look at the GSoC archive. But, sometimes term will change as the organization announced. Some organization will participate again, while others not. So, for me it better to prepare after the program announced officially.
From there, I can see which project fit with my programming language, one that I love. So, after some effort of filtering and analyzing, then I choose an appropriate project, I will not find a big technical barrier to pass this program.

Pick a project that fit with my knowledge

This means, I must look at the project task. With given time frame, which about just 3 month, does it sound realistic that I will finish my task? That it’s, be realistic is the key. Measure the difficulty of the task. When I face something very unfamiliar and need more time to learn, I prefer to skip this. Because I don’t want to spend my time in things that I less understand.
For example, this year, I choose project from GNU Wget that use C as their programming language base, which currently I am familiar with. And for the task, I pick idea of “Improvement of test suite using Libmicrohttpd”.

Get enganged with community

After I found the project that I want to apply, I must introduce myself to the community. This is important step I must do. Because without it, mentor cannot help me when I face difficulty.
Find the place where they collaborate. There are some choice like mailing list, slack, gitter, irc, etc. GNU Wget use mailing list as they medium to communicate between developer. I send welcome message there, and replied kindly by other members. Follow their rules. Basically, there are some rules to be obeyed: - Use plain English. GSoC is global program and use English as it language for communication. So, I need to use English in my conversation. - Not to send message directly to the mentor or other person in the community. This is considered to be rude. Instead, I will use mailing list if I need to send a message. - Avoid top post when I reply an email.

Learn their environment, workflow and code base

After I “Say Hi!” to the organization, now it’s time to dive into their code. Find the repository. In Wget, their main repositories are hosted in Savannah, Github and Gitlab. So, I clone the repo and then build the software from source.
I have successfully built Wget from source. Then I try to trace what the program do using gdb, to take a look what is going on inside its process.
I familiarize my self with git, because Wget use git as its version control system. Also the project uses Github to consolidate patches, open issue, write wiki, etc. I try to hanging around on there. It fancy design keep me comfort with environment and workflow of the project.

Submit a patch, event with a trivial one

This patching section required quite technical skill. For newcomer like me, I feel a bit confuse about what patch I can submit. Then, with guide from my mentor, I was given an idea what simple task can be submitted. I pick an easy one. A testing unit. Because, he mention, that Wget unit testing is still far from perfect. I will take it for my chance. I try to add a function to reach more coverage in the project unit test.
In order to merged into upstream, I need to make pull request on that project. To be honest, it is my first time pull request in a real open source project. Of course my patch will not merged automatically to the upstream. Mentor and the community members will review first before it can be accepted. And after several step, edit code, re-push commit with guide from my mentor and other contributors that help me, finally my patch get merged on the upstream. It’s a nice feeling for a rookie developer can have.

Write a good proposal

After all, to be accepted in the GSoC program itself, I need to upload a proposal. The proposal must be good enough to be considered accepted as GSoC candidate. I pick the idea from list project that mentioned in project website. I create a draft from this and share to other members. I receive a lot of feedback. Thanks for who was concerned, so I can make improvements to my proposal. After I feel it ready, I upload my final proposal to GSoC official.

That was my short list for what I do in this GSoC. And because the time between the date of proposal submit and the announcement of accepted applicant is a month long, I will use it to get more familiar with project code base.

Convert Text to Pdf on Linux

Background: failed to build linux kernel documentation (still too lazy to solve problems).
Goal: generate pdf files of linux documentation, because I have reading problem when using text files. Text files look ugly when I open using my reader (Android).

Install needed packages.

$ sudo apt-get install enscript ps2pdf

Do this simple tricks.

$ cd Documentation/
$ for i in `ls -p | grep -v /` ; do ; enscript -p$i.ps $i ; done
$ for i in `ls *.ps` ; do ; ps2pdf $i ; done

Now I have pdf files of all the text files.
It still have a problem, it just convert files on the same directory. So other files under this directory must be processed separately, which mean I must move to each directory.

My Vim Configuration

This is my Vim configuration a.k.a .vimrc. You can download the file from here. Feel free to use and customize.

Below are some description of its contents:

Use vim settings instead of vi

set nocompatible

No backup or swap

set nobackup nowritebackup noswapfile autoread

Enable per-directory .vimrc files

set exrc

Don’t unload buffer when switching away

set hidden

Allow per-file settings via modeline

set modeline

Disable unsafe commands in local .vimrc files

set secure

Saving and encoding

set encoding=utf-8 fileencoding=utf-8 termencoding=utf-8

Command completion

set wildmenu

Allow backspacing over everything in insert mode

set backspace=indent,eol,start

Display status line which contains current mode, file name, file status, ruler, etc.

set laststatus=2

Always set autoindenting on

set autoindent

Display incomplete commands

set showcmd

Keep upto 50 lines of command line history

set history=50

Show a vertical line at the 79th character

set textwidth=80

Highlight column after ‘textwidth’

set colorcolumn=+1

Switch syntax highlighting on

syntax on

Switch highlighting on the last used search pattern

set hlsearch incsearch ignorecase smartcase

Don’t hide the mouse cursor while typing

set nomousehide

Right-click pops up contect menu

set mousemodel=popup

Show cursor position in status bar

set ruler

Show line numbers on left

set number

Disable code folding

set nofoldenable

Scroll the window so we can always see 10 lines around the cursor

set scrolloff=10

Kernel coding style

set tabstop=8                                           
set softtabstop=8                                       
set shiftwidth=8                                        
set noexpandtab

Enable file type detection

filetype plugin indent on

Shortcut to search visually selected text

vnoremap // y/<C-R>"<CR>

Check if running on gvim

if has("gui_running")

Set terminal color to 256

set t_Co=256

Check if using Windows

if has("win32") || has("win16")

Set Ubuntu Mono font with size 11

set guifont=Ubuntu\ Mono:h11

Use solarized colorscheme

colorscheme solarized

Set the background to dark color

set background=dark

Highlight the current line

set cursorline

Use letter as the print output format

set printoptions=paper:letter

I use Ubuntu font which can be download from font.ubuntu.com. To enable solarized colorscheme, download file solarized.vim from https://github.com/altercation/vim-colors-solarized/. If you want to look my personal vim configuration, you can find it here.

I use this configuration in my Linux workstation and server. I use exclusively for editing linux kernel and other open source projects. For daily programming use, you can change the value of tabstop, softtabstop and shiftwidth.
Happy Vimming!

Vim Quick Reference

For unfimiliar user, vim is a little bit confusing at first. But, after you try it several times and configure properly, you will found this text editor is very powerful.
Here I will note several subject how to use vim, because sometimes navigate to vim help is quite difficult.

Vim is modal editor which means you use different mode to edit text. Some important mode you will use very often are:

  • Normal Mode

You can enter any command using : or search using / and ?. Press Esc to enter this mode.

  • Insert Mode

Text you type is inserted into the buffer. Press i to enter this mode.

  • Visual Mode

In this mode, you can select the text we want in order to copy, cut or delete. Press v to enter this mode. The cursor position will be starting point to your selection.

Navigation

You can move through the buffer with the following key:

Move left h
Move right l
Move up k
Move down j

Using this key will make your navigation faster than using arrow keys or mouse.

Move one word w
Move one word backwards b
Move to end of line $
Move to beginning of line 0
Insert text i
Insert text at the end of line A
Insert text at the beginning of line I
Insert text above line O
Insert text below line o
Replace one character r
Replace character consecutively R (enter “replace mode”)

Page navigation:

Move page down ctrl + f
Move page up ctrl + b
Move page half down ctrl + d
Move page half up ctrl + u
Move top gg
Move bottom G

Navigate through command history q: Navigate through search history q/ Then you can select command or search result with navigation key. To execute command, press Enter.

Open and Save File

Open file :e filename
Save file :w
Saveas file :w filename
Quit :q
Quit without save:q!
Save file and quit:wq

Undo and Redo

Undo u
Redo ctrl + r

Visual Selection

Use v on text you want to select. You will enter visual mode. Navigate using hjkl key. Then we can do other action such as copy, cut or delete.

Copy and Paste Text

After you select a text, you can do following action:

Copy y
Cut/Delete d or x
Copy entire line yy
Cut entire line dd
Paste p

Search and Replace Text

Find text /query
Find text upwards ?query
Navigate between search results n to move forward and N to move backwards

Find text using visually selected text, firstly add this line to vimrc file

vnoremap // y/<C-R>"<CR>

then select text in visual mode and press //

Replace text :%s/old/new/g
Replace all :%s/old/new/gc

Word Completion

You can autocomplete text with several method to get different result:

Search text before cursor ctrl + p
Search text after cursor ctrl + n
Search for filename complete with its path ctrl + x ctrl + f

Multi Window Editing

Open new window :new
Open new vertical window :vnew
Move through window ctrl + ww
Swap window position ctrl + wr

Compare Buffer

If you have two split windows containing buffers that you want to compare, then you can diff them by running :windo diffthis
You can turn diff mode off just as easily, by running :windo diffoff

That’s how I use vim in daily life. For my personal preference, I have modified my vimrc which you can find here. Thanks for reading.