Shankar Chakkere's Blog

Some time you need a binary file with known contents to test your communication software.

Here is a quick way to generate one using Gawk. Gawk is GNU AWK.

/cygdrive/c/develop> gawk 'BEGIN {x=0; while(++x<=255){printf "%02x ",x;}; exit}' |xxd -r -p >numtable.bin

Creates a binary file which contains value from 0 to 0xff.

/cygdrive/c/develop> od -t x1 numtable.bin
0000000 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10
0000020 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20
0000040 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30
0000060 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 40
0000100 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50
0000120 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 60
0000140 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70
0000160 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 80
0000200 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f 90
0000220 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f a0
0000240 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af b0
0000260 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf c0
0000300 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf d0
0000320 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df e0
0000340 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef f0
0000360 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff
0000377

You can extend this to generate a PROM/ROM lookup table like for example a sine table for an 8 bit table.

/cygdrive/c/develop> gawk 'BEGIN {x=0;pi=3.14; while(++x<=90){printf "%02d ",sin(pi/180*x)*100;}; exit}' 

Labels: Utilities

It is difficult to keep track of all the passwords. I used to write passwords in Kannada ,my language, hoping that not many can read. Now I save all my passwords using KeePass on Windows and KeePassX on Mac OS X. It keeps the entire password in a database and locks it with one master password. The database is compatible across the platform, and the same database can be used on Windows or Mac OS X.

The problem:
Imagine you are on the road, you can't recall the password for a certain web site.
Here is the solution:
Save a password database as text file and encrypt that file.

Step 1. In keeppass File -> Export To -> CSV File (Under Windows, Or as Text in Mac OS X).
Save it as say Database.kdb.csv. Copy this file onto your thumbdrive.

Step 2. Compiling a multiplatform file encrytion program bcrypt.

In Windows(cygwin):

Download the source and unpack into a directory.

schakkere@shankar ~
$ cd develop/bcrypt/
/ecos-c/usr/shankar/develop/bcrypt

schakkere@shankar ~/develop/bcrypt
$ make
gcc -O2 -Wall  -c main.c
gcc -O2 -Wall  -c blowfish.c
gcc -O2 -Wall  -c rwfile.c
rwfile.c: In function `deletefile':
rwfile.c:123: warning: implicit declaration of function `initstate'
gcc -O2 -Wall  -c keys.c
gcc -O2 -Wall  -c wrapbf.c
gcc -O2 -Wall  -c endian.c
gcc -O2 -Wall  -c wrapzl.c
gcc -O2 -Wall  -o bcrypt main.o blowfish.o rwfile.o keys.o wrapbf.o endian.o wrapzl.o -L/usr/local/lib -lz
Info: resolving _optind by linking to __imp__optind (auto-import)
Info: resolving _optarg by linking to __imp__optarg (auto-import)

schakkere@shankar ~/develop/bcrypt
$ make install
mkdir -p /usr/local/bin;\
mkdir -p /usr/local/man/man1;\
cp bcrypt /usr/local/bin;\
cp bcrypt.1 /usr/local/man/man1;\
chmod 755 /usr/local/bin/bcrypt;\
chmod 644 /usr/local/man/man1/bcrypt.1

schakkere@shankar ~/develop/bcrypt
$ which bcrypt
/usr/local/bin/bcrypt

schakkere@shankar ~/develop/bcrypt
$ bcrypt
Usage is: bcrypt -[orc][-sN] file1 file2..
-o Write output to standard out
-r Do NOT remove input files after processing
-c Do NOT compress files before encryption
-sN How many times to overwrite input files with random data

To encrypt a file on your thumb drive.

schakkere@shankar ~/develop/bcrypt
$ cd /cygdrive/E/backup/keepass/

schakkere@shankar /cygdrive/E/backup/keepass
$ ls
Database.kdb  Database.kdb.csv

schakkere@shankar /cygdrive/E/backup/keepass
$ bcrypt -s5 Database.kdb.csv
Encryption key:Key must be at least 8 characters
Encryption key:
Again:

schakkere@shankar /cygdrive/E/backup/keepass
$ ls -al
drwxr-xr-x  2 schakkere mkgroup-l-d     0 Feb  7 00:21 .
drwxr-xr-x 11 schakkere mkgroup-l-d     0 Feb  7 00:20 ..
-rw-r--r--  1 schakkere mkgroup-l-d 13084 Jan 10 12:23 Database.kdb
-rw-r--r--  1 schakkere mkgroup-l-d  2126 Feb 15 22:11 Database.kdb.csv.bfe

Option -s5 overwrites the original file 5 times with random data before deleting it.

Copy bcrypt.exe and cygwin1.dll onto your thumb drive

In Mac OS X:
Unzip the source file.

Shankar-Chakkeres-Computer:~/develop/bcrypt shankar$ make

Copy bcrypt to ~/bin and onto your thumb drive.
You can also run make install if you want it to be available for other users on your system.

Testing it:
Plug the thumbdrive

Shankar-Chakkeres-Computer:~/develop/bcrypt shankar$ ~/bin/bcrypt -r -o /Volumes/6JAN2007/backup/keepass/Database.kdb.csv.bfe | grep -i bank
Encryption key:
"Bank of America","xxx","xxx","xxx","xxx"

It works!

Using it on the road:
Step 3. So now you can use your password on the road even on the machines which does not have bcrypt installed (because you have the executable for Windows and Mac OS X and the encrypted password file on the thumb drive).

Labels: HOWTO, OS X, Utilities

Do you wish to convert your vacation memory captured in a Camcorder into a DVD?

Here are the steps involved in making it with Free/Open Source software on Windows and Mac OS X:

1. Get the video stream into the computer.

It may be as simple as connecting the camcorder to the computer
Capturing the analog out of the camcorder using a video grabber card
Through a PVR (I have replayTV).

The captured stream may be in MPEG or DV format.

2. Editing the video stream.

I use mpegstream to edit the video. It supports the entire format Apple's QuickTime supports.

Here is the list of supported input formats:

MPEG, VOB, PS, M2P, MOD,VRO, DAT, MOV, DV,AVI,MP4, TS, M2T, MMV,REC, VID, AVR, M2V, M1V, MPV, AIFF, M1A, MP2, MPA, AC3

It is multiplatform (OS X and Windows) free tool. It can export the edited stream into many formats, so that it becomes easy to import on your DVD Authoring tool. If I am using DVDStyler (see STEP 3) I demux the ouput into M2V (Video) and MPA (Audio).

If you are using Mac OS X, you can use iMovie to edit the stream. The stream should be in DV or MPEG-4 format.

3. Authoring the DVD.

In windows I use DVDSTyler an open source DVD authoring tool. It runs on Linux too. Use DVDStyler to create buttons, chapters. DVDStyler cannot burn a DVD; it creates an ISO image of the DVD. If you have Nero you can mount the ISO image and verify your creation before committing on a DVD!

On Mac OS X I use iMovie and iDVD for authoring and burning.

4. Burning the DVD from the ISO Image.

In windows I use imgburn a free tool for burning an ISO image. If you have Nero you can use it too.

Labels: HOWTO, Utilities

Like many I use thumb drive. I store personal information in it. I want to protect that information even if I lose that drive. I use TrueCrypt, an open source cross platform disk encryption software. It creates an encrypted file system on a file with a password. When I mount this file, it appears as a drive.

In the above picture it appears as drive W: You can use it as a normal drive.

When I am done I will unmount it, also if the drive is not used for long time it unmounts itself. TrueCrypt has many other features like Traveler Mode. There is no Mac OS X port yet.Mac OS X version of TrueCrypt.

Labels: Utilities

Did you accidentally erased photos/files from compact flash disk? Recuva can recover these deleted files. Recuva is a freeware.


It goes without saying that the flash disk should not have been used after erasing to recover the files.

Note: Most of the camera uses FAT file system. FAT file system just changes the first character in the file name to a special character, without deleting the file content.
Technorati Profile

Labels: Photography, Utilities

If you represent anything graphically, it becomes easy to understand. If the directories and files are represented by blocks whose size depends on size they occupy! You can instantly see which directory or file is a space hog. This graphical representation of things is called treemap

There are many utilities which use treemap to show sizes of files and folders.
Here are the ones which I use:

With OS X
Disk Inventory X


With Windows
Windirstat


For Linux/KDE
Kdirstat

Labels: Utilities

There are various ways to generate drawing for documentation. For me things are easier if I do not use GUI and Mouse unnecessarily.Graphviz is open source graphic visualization software developed by AT&T. It runs on multiple platforms, I have used it in Windows and Mac OS X.

It accepts scripts written in dot language and produces a graph/picture. Here is one such drawing for documentation.

---

---

The dot file (script) which generates the above drawing.

file: commandstructure.dot

digraph structs
{
node [shape=record];
struct1 [shape=record,
label=" COMMAND| \
    COMMAND_TYPE| \
    COMMAND_SUB_TYPE|\
    PARAMETER"
];
struct2 [shape=record,
label=" {\
  {0x1|STRT}| \
  {0x2|MONI}| \
  {0x3|RPLY}| \
  {0x4|PLYD}\
  }"
];
struct3 [shape=record,
label=" {\
  {0x0|NONE}| \
  {0xX|CHNL}| \
  {0xX|BUTN} \
  }"
];
struct4 [shape=record,
label=" {\
  {0x0|OFF}| \
  {0x1|ON }| \
  {0x2|STP} \
  }"
];
struct1:f1 -> struct2;
struct1:f2 -> struct3;
struct1:f3 -> struct4;
}

The command to generate the diagram.

c:\usr\shankar\develop\graphviz>dot -Tpng commandstructure.dot -o command.png

Note the diagram is generated in png format. It can also render it in many other formats like: canon, cmap, cmapx, dia, dot, fig, gd ,gd2, gif, hpgl, imap ,ismap, jpeg, jpg, mif, mp, pcl, pi,ps, ps2, svg, svgz, vrm, vtx, wbmp, xdot. So generate in the format your documentation tool can import.

Graphviz is used by Doxygen to draw C++ class diagrams, dependency graphs etc. of a project. I will cover it some other time.

Labels: Utilities

There are multiple extensions which you can add to firefox to select and download multiple files from a site.I like to do this easier way on a command line.

In Windows:

$ dog --links http://blogs.parachute.com/jobtalk/archives/podcasts/index.html |grep mp3|sed -e "s/^/ wget -c /" >gett.bat

In Mac OS X:

$~>echo '#!/bin/sh'  >gett.sh
$~>dog --links http://blogs.parachute.com/jobtalk/archives/podcasts/index.html |grep mp3|sed -e "s/^/ wget -c /" >>gett.sh
$~>chmod +x gett.sh

Explanation:
dog gets the list of links in the URL. dog is better than cat(concantenation)!
grep selects the URL of only the mp3 file.
sed adds wget -c before each line.

For Windows:
wget, grep and sed are available through Cygwin. You can compile dog in Cygwin environment. Double clicking the batch file gett.bat will download all the files you selected.

For Mac OS X:
wget, grep and sed are available through terminal. You can compile dog and install. Issue $~>./gett.sh to download the files selected.

You can download the source for dog from here: http://ftp.kernel.pl/people/baseciq/rpm/src/dog-1.7-1.src.rpm. The source is in the rpm(Redhat package format). I used Total Commander to extract the files in Windows.

Labels: Utilities

While testing a codec I needed to find out how many milliseconds of audio is in the encoded file.I decode this data into Wave format.

The encoded file.

c:\usr\shankar\lib>cat packed.pcm |od -h |head
000000 6b21 0fc5 0457 6f00 065d 58c8 6b21 0dc5
000010 0c3b de36 a172 8a89 6b21 0dc5 cc97 ef2c
000020 8381 bc56 6b21 0b63 849b 300a 4b8c 927b
000030 6b21 0f65 5c3b bb40 c56d 7567 6b21 0dc5
000040 943b a645 8869 94a9 6b21 0d45 143b 7314
000050 8469 f256 6b21 7f75 1c3b 1fa6 e571 22c7
000060 6b21 cb63 5c3a 5708 796c 2a94 6b21 6f75
000070 043b 1d27 1171 2201 6b21 0d45 e43b 9735
000080 7c6f 9e03 6b21 7f75 c43b 1f32 3875 a76c
000090 6b21 0dc5 f437 e5ad f36c b14f 6b21 cb63

Ten millisecond worth of encoded data is delimited by 6b21. So if I count the number of 6b21's in the encoded data, I get the total number of millisecond.

Here is one way to do it.

c:\usr\shankar\lib>cat packed.pcm |od -h |cut -d' ' -f2-| tr `\ ``\n` |grep 6b21 |wc
300    2700   14100

There is 3 Sec (300 *10 millisecond) of audio in this file.

Explanation:
The command cut -d' ' -f2- cuts the first column which is address field
The tr command translates space to linefeed as shown below.

c:\usr\shankar\lib>cat packed.pcm |od  -h |cut -d' ' -f2- |tr '\ ' '\n'|head
6b21
0fc5
0457
6f00
065d
58c8
6b21
0dc5
0c3b
de36

Why in this format?
Because I just have to extract 6b21's using grep as shown below

c:\usr\shankar\lib>cat packed.pcm |od  -h |cut -d' ' -f2- |tr '\ ' '\n'|grep 6b21 |head
6b21
6b21
6b21
6b21
6b21
6b21
6b21
6b21
6b21
6b21

Now wc will only count 6b21's!. All the above unix commands are available for Windows by installing Cygwin or Djgpp.

Labels: script, Utilities