Git Online Books

Git Community book
Pro Git
Git - SVN Crash Course

Get list tables from postgres database

If we need to iterate over all tables of a certain schema, we can get the list of schema, table, type and user using psql:

$ psql -qAtF, -c "\dt da_schema.*"
da_schema,segments,table,jacen
da_schema,speeds,table,jacen
$ _

You can also use "*" in the schema name, for example:

$ psql -qAtF, -c "\dt da_schema*.*"
da_schema,segments,table,jacen
da_schema,speeds,table,jacen
da_schema_2,labels,table,jacen
da_schema_test,positions,table,jacen
$ _

Changing execution attributes on svn

Sometimes a script is added to a SVN repository without execution permissions. After that, a simple chmod doesn't work for SVN.

This is what can be done to change the attributes of the file:

$ svn propset svn:executable on da_scripto.sh
$ _

Rebuilding TeX's filenames database

I had some problems with Beamer installation (http://latex-beamer.sourceforge.net). After install this TeX class, I couldn't compile a tex file, dumping this error:

! LaTeX Error: File `beamerthemeAnnArbor.sty' not found.

The solution was simple, rebuild the TeX's filenames database

sudo texhash

Attach a file to a mail on shell

In the old times we were able to attach a file using the mail command. Now, we can do the same thing using mutt:

$ echo "Mail body" | mutt -s "This is a mail with attachments" -a ~/mi_file.txt -a ~/mi_picture.png anne@octop.us
$ _

We can also use some pre-written file to use as the email body:

$ mutt -s "Mail with body from a file" anne@octop.us < ~/mail_body.txt
$ _

Work with Data/Time in Postgres

A common scenario in Postgres (at least for me) is do conversions between different time formats. For example

* How to transform 'time' from epoch to any human readable format?

  SELECT to_char(to_timestamp(1239148800) at TIME ZONE 'UTC', 'YYYY/MM/DD HH24:MI:SS')

For any other format, you can see the table "Template Patterns for Date/Time Formatting" on the documentation of your version of Postgres.


* How to transform from timestamp to epoch ?

  SELECT EXTRACT (EPOCH FROM timestamp '20090101T041000' at time zone 'UTC') ;

Other way to do the last transformation is set the time zone on the environment

  SET TIMEZONE TO 'UTC'
  SELECT EXTRACT(EPOCH FROM timestamp '20090506T041000');

Generating lines from one line

If you want to generate lines from one line according to a field

You can do in perl with:

$ echo "1,a/b/c/d" | \
perl -F, -l0ane 'print "$F[0],$_\n" foreach(split("/",$F[1]))'
$

Or if you want to write less do:

$ echo "1,a/b/c/d" | awk -F, '{split($2,z,"/");for(i in z)print$1","z[i]}'

Get IP address of a host

If you need to get the IP address of a host, but you don't want to parse the output of ifconfig, you can use hostname:

$ hostname -i
192.168.1.160
$ _

UnicodeEncodeError: 'ascii' codec can't encode character XXX

 Yet another text-format error in python (personal error, of course)...  while I was trying to store in file a joined list (through list comprehension), one or more of the characters of one the fields that are being joined generate the following 'friendly message':

Traceback (most recent call last):
  File "stations.py", line 62, in
  exit(main())
  File "stations.py", line 16, in main
  record.append(str(element.text))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 7: ordinal not in range(128)

 to avoid it, instead of trying to cast into a string the variable, you could use the repr() function... 

repr(object)
  Return a string containing a printable representation of an object.
  (http://docs.python.org/library/functions.html)

Example:

>>> caca = u'\u2603'
>>> print caca
☃
>>> str(caca)
Traceback (most recent call last):
  File "", line 1, in
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2603' in position 0: ordinal not in range(128)
>>> repr(caca)
"u'\\u2603'"

solution from:
http://blog.codekills.net/archives/45-str...-yer-probably-doin-it-wrong..html

shp2pgsql UTF-8 enconding workaround

While trying to convert a shapefile and load it to a DB, some latin characters on the data were a problem and the parameter that shp2pgsql provides for the encoding did not work

shp2pgsql -W "UTF-8" ... :(

Then, a nice workaround is to use | with iconv:

shp2pgsql  file.shp  schema.table | iconv -f LATIN1 -t UTF-8 | psql

Python broken pipe

This could be little weird but I have seen this, more than once. When you work in python , parsing the standard input, and for some reason you use a pipe at the end of you process to see the head or tail like:
python foo.py| head
and you get a Broken Pipe Error then the next code snippet could be useful.

import sys
for line in sys.stdin:
  print line.strip()

Then run your program and you will get

$ yes | python pipe.py |head
y
y
y
y
y
Traceback (most recent call last):
  File "pipe.py", line 5, in ?
    print line
IOError: [Errno 32] Broken pipe

To avoid this error just use signal package:

import signal
import sys

signal.signal(signal.SIGPIPE, signal.SIG_DFL)

for line in sys.stdin:
  print line.strip()

Enjoy

Error handling in Bash

To check if a command was successfully run in our bash script, we have to check the value of its output, which is stored in the $? variable. If its value equals to 0, then the command was successful, otherwise, an error was thrown:

rm ~/data.tar.bz2
if [ "$?" != 0 ]; then
   echo "ERROR: Could not delete file"
   exit 1
fi

$ _

The kill that revives

Let's say your working on a remote console and send a process to sleep (CTRL-Z), all of the sudden your connection freezes and you loose the ability to interact with the slept process and bring it back to life. Don't connect again to kill it and restart it. Connect again to revive it with kill

$ kill -SIGCONT pid
$_

:)

disown ...

For the bash lovers:

It happens to all of us ...
You launch a command that is taking too long, you can't logout 'cause you'll lose your session and all the running hours...

what to do??

$ ./myprocess.bash
^Z
[1]+  Stopped ./myprocess.bash
$ bg
$ disown %1
$ logout
$ _

Now INIT will take care of your son, and you are free to leave ...
(see "man disown", a bash built-in).

Adding en_US.utf8 locale to an Ubuntu

sudo localedef –no-archive -i en_US -c -f UTF-8 en_US.UTF-8


locale -a

C
en_US.utf8
POSIX

For the next time you log into the server the following two lines have to vi added to /etc/environment

LANGUAGE=”en_US.utf8″
LANG=”en_US.utf8″

double/single quotes in bash

Single Quotes (' '): Preserve the literal value for each character within the quotes. For example,

$ string="Hello World"
$ echo $string
$ echo '$string "string"'
$ _

Double Quotes (" "): Preserve the literal value of all characters within the quotes. Rules Exception are: ‘$’, ‘`’ and‘\’ (shell expansion). For example,

string="Hello World"
echo $string
echo "My first script: $string, \"string\", "string""
$ _

Back Quotes or Back Sticks(``):Command Substitution (idem $() )
For example,

echo `date`, $(date)
$ _

Adding Google projection to your PostGIS

INSERT into spatial_ref_sys (srid, auth_name, auth_srid, proj4text, srtext) values ( 900913, 'spatialreference.org', 900913, '+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs', 'PROJCS["unnamed",GEOGCS["unnamed ellipse",DATUM["unknown",SPHEROID["unnamed",6378137,0]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]],PROJECTION["Mercator_2SP"],PARAMETER["standard_parallel_1",0],PARAMETER["central_meridian",0],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["Meter",1],EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs"]]');

Files being accessed by a process

$ ls -l /proc/pid_number/fd_

Try with the pid of your firefox process and you'll see all the sqlite files where firefox stores places, cookies, downloads, permissions, etc.. All the jar files, certificates, extensions and more.

Try:

$ echo ".dump" | sqlite3 /path_to_firefox_profile/formhistory.sqlite

:P

Request Layer with another SLD from OpenLayers

Reports is the layer name in the map and SLD is the sld name

layer=Map.getLayersByName('Reports')[0];
layer.params['STYLES']='SLD';
later.redraw();

Kill'em all >:|

$ kill -9 -1
$ _

Kill all processes owned by the user except the shell from where it's being launched, it's a nice cleanup shortcut.

Code Highlighting

Now you can post your snippets codes with a nice highlight.
Example:

   from datetime import datetime
   print datetime.now()
 

To use it, just write your code inside of <pre> or <textarea>
with tags name="code" and class="language" where language could be:

• Javascript (js)
  
• Bash (bash)
  
• Python (python)
  
• Sql (sql)
  
• C++ (cpp)
  
• Java (java)
  
• Php (php)
  

If you are looking for a console style, use class="console", and <blink>_</blink>
for your cursor :)

How many fields has my CSV file?

$ head -1 my.csv|tr -cd , | wc -c
$ _

(add 1 to the result)

dr.chamberlain:

$ head -n1 my.csv|awk -F, '{ print NF }'
$ _

Expanding strings in shell

$ a="my string"
$ _

Simple quote doesn't expand the variable:

$ echo 'This is $a'
This is $a
$ _

Double quote does expand the variable:

$ echo "This is  $a"
This is my string
$ _

Paico

$ printf 8;printf "=%.0s" {1..15};echo D _

list processes with open sockets

lsof -i -r30 -n +c0

-r30: list every 30 secs
-n: do not resolve names
-i: sockets
+c0: do NOT truncate chars

The only time bash has been unable to help me.

$ [ Where is my brain?
bash: [: missing `]'

Days between 2 unix epoch

#!/bin/bash
# Calculate "days diff" (unix epoch)
function daysdiff {
    begin=${1:-0}
    end=${2:-0}
    let Sec=$end-$begin

    if ((Sec < 0)); then
      echo $((Sec/-86400))
    else
      echo $((Sec/86400));
    fi
    return 0
}

Mount a directory through ssh

# Mount as read only directory from
# remote host (username@host:/dirnamehost) to
# local host (dirnamelocalhost)

sshfs -o ro username@host:/dirnamehost dirnamelocalhost


# remember that the user must belong to the fuse group
# before doing the sshfs
usermod -a -G fuse username

Working with date in shell

If you don't know what time is the unix time 1245257034

<br /><br />$ date -d @1245257034<br />Wed Jun 17 12:43:54 EDT 2009<br /><br />

Or in UTC

$ date -d @1245257034 -u
Wed Jun 17 16:43:54 UTC 2009

To know what is the current unix time in your machine

$ date +%s
1245257175

Getting last part of a string in shell script

# If we have the string

string="my string.hello.world"

# to get the last part considering the field separator = "." execute:

echo ${string##*.}