Robert Lujo's Homepage

Answering questions on Stackoverflow

Sun, 12 May 2013 23:25:00 +0200

The last few days I was answering questions on stackoverflow - mainly related to python and db2. It came out a bit addictive. Now I have 20 answers, raised my reputation to 449 and reached top 10% this week :).

Check my profile at http://stackoverflow.com/users/565525/robert-lujo .

Destructuring assignment / tuple unpack in Python - by examples

Fri, 18 Jan 2013 23:33:00 +0100

Destructuring assignment (a.k.a. tuple unpack) in Python, seems to be very useful thing. The most basic variant:

»> x,y = 1,2
»> x,y
(1, 2)

what is identical to:

»> x, y = [1,2]
»> x, y = (1,2)

Any iterator would fit, e.g. class impl. __iter__ function (coroutine):

»> class test(object):
… def __iter__(self):
… for x in (1,2): yield x
»> a,b = test()
»> a,b
(1, 2)

regular function would produce the same result:

»> def test():
… for x in (1,2): yield x

Using destructuring is specially useful when function/method returns more than one result (technically, it returns only one result - tuple):

»> def f(): return 1,2
»> x,y = f()
»> x,y
(1, 2)

Although I’m working with python from the version 2.2., recently I have played around with Clojure and discovered that some advanced desctructuring features that Clojure offers, can be found in python too, e.g.:

»> x,[y,z] = 1,[2,3]
»> x, y, z
(1, 2, 3)

more complex example (note that variable “_” is 2 times assigned):

»> x,[y,z,[w,_,t,_]] = 1,[2,3,(4,5,6,7)]
»> x,y,z,w,t,_
(1, 2, 3, 4, 6, 7)

What looks very useful is that, destructuring can be used in function’s argument declaration:

»> def f(a,(b,c)):
… print a,b,c

This function expects 2 arguments, of which second one must be some iterator type with exactly two members:

»> f(1,(2,3))
1 2 3

——————————

UPDATE:It seems that python3 deprecates this feature, i.e. this doesn’t work in python 3.1:

def f(a,(b,c)): print a,b,c

Why? Read http://www.python.org/dev/peps/pep-3113/. Any alternative? They suggest using function annotations (py3+) http://www.python.org/dev/peps/pep-3107/.

——————————

Python 3.0+ offers taking rest of tuple into last variable:

»> x,y,*z = 1,2,3,4,5
»> x,y,z
(1, 2, [3, 4, 5])

more complex example - having nesting lists (python 3.0+):

»> x,[y,*z],t,*w = 1,[2,3,4],5,6,7
»> x,y,z,t,w
(1, 2, [3, 4], 5, [6, 7])

Argument unpacking naturally follows destructuring theme, which can be seen as “opposite” operation . Small example:

»> def f(a,b): print a,b
…
»> f(*[1,2])
1 2

(Note: the same could be achived with apply function until it was deprecated in python 3+).

Destructuring is common thing in other languages, e.g. javascript, ruby, php (example http://artlung.com/blog/2007/02/21/destructured-assignment-in-ruby-php-and-javascript/).

It seems that good ideas are constantly being freely “borrowed”.

permset - utility for *nix permissions mgmt based on patterns - release 0.22

Sat, 15 Dec 2012 10:02:00 +0100

Permset is simple standalone utility script to manage *nix permissions on file and directory trees based on patterns. New release 0.22 contains following improvements:

variable substitution for user and group in patterns
short statistics displayed on the pattern listing
documentation improvements

permset - utility for mgnt of *nix permissions based on patterns

Tue, 11 Dec 2012 18:34:00 +0100

I had just released 0.20 version of permset.

Permset is simple standalone utility script to manage *nix permissions on file and directory trees based on patterns.

Check the example session to get into the idea behind.

Twitter incremental backup in YAML format - python, HTML get and parse

Sat, 24 Sep 2011 19:35:00 +0200

I use Twitter primarly as a bookmarking service.

I know that Twitter is not bookmarking service and there are specialized services for such needs. In my case, using this way started as an experiment, but later I found this way very convenient.

I wrote a script that enables me to incrementally backup all my tweets in a textual file in YAML format, by parsing HTML Twitter pages. Explanation and the script follows …

My case

So this is my case:

I read a lot
Google Reader
HTC Desire smartphone
everything I found interesting is sent to Twitter by using Android Peep Twitter application
tweets are tagged by prefixing keywords with hashes (#)
tweets are backuped to textual file which is held in my Dropbox folder
backup should be incremental
backup should be readable by human and machine - YAML is chosen
(future) all tweets/bookmarks will be publicly available on my site with advanced browsing feature

The script

Based on the script made by Scott Carpenter I made a python script that doesn’t use Twitter API, but reads Twitter HTML pages, parses them and collect tweets. On the other hand backup file is in YAML file, it is parsed and then only collected tweets that don’t exist in YAML file are written to the top of the file. The script needs BeautifulSoup HTML parsing package, and pyyaml so if you don’t have it yet:

pip install beautifulsoup
pip install pyyaml

Example of usage

Example of usage:

> dbox twitter-bak trebor74hr twitter-trebor74hr-backup.yaml
Reading http://twitter.com/trebor74hr, backup in twitter-trebor74hr-backup.yaml
started at 2011-09-24 18:30:13.937000
  1. page - read and parse
      20/ 20 tweets saved/processed. Waiting 2 seconds before fetching next page...
  2. page - read and parse
      40/ 40 tweets saved/processed. Waiting 2 seconds before fetching next page...
  3. page - read and parse
      60/ 60 tweets saved/processed. Waiting 2 seconds before fetching next page...
  4. page - read and parse
      80/ 80 tweets saved/processed. Waiting 2 seconds before fetching next page...
  5. page - read and parse
      89/100 tweets saved/processed. Waiting 2 seconds before fetching next page...
  6. page - read and parse
      No new tweets found, quit iteration ...
89/120 tweets saved in twitter-trebor74hr-backup.yaml

Try again, no new tweets:

> dbox twitter-bak trebor74hr twitter-trebor74hr-backup.yaml
Reading http://twitter.com/trebor74hr, backup in twitter-trebor74hr-backup.yaml,
started at 2011-09-24 18:30:49.906000
  1. page - read and parse
     No new tweets found, quit iteration ...
No new tweets found in 20 tweets analysed, file twitter-trebor74hr-backup.yaml

The result

The result is in twitter-trebor74hr-backup.yaml:

- content : "RT @williamtincup RT Don’t Be 'That Guy' as a Manager
            bit.ly/qJL8nx [http://t.co/fxy2IrIe] @greatleadership"
  date_time: "Fri Sep 23 20:17:32 +0000 2011"
  status_id: "117331787797102592"
  url: "http://twitter.com/#!/trebor74hr/status/117331787797102592"

- content : "python.mirocommunity.org [http://t.co/WYu9ygAb] #Python
            #Miro  Community - All Python #Video , All the Time"
  date_time: "Fri Sep 23 20:12:34 +0000 2011"
  status_id: "117330534635540481"
  url: "http://twitter.com/#!/trebor74hr/status/117330534635540481"

...

Again, the script is …

The script can be found here.

Mercurial branches - 'named branches' way - prefered way to do it?

Thu, 08 Sep 2011 23:33:00 +0200

Mercurial provides many ways to do branch based development. I was searchingfor the best in-repo branch solution. I chose named branches …

Branching in mercurial

Mercurial provides following solutions to branch based development:

hg tag + hg clone + push/pull :
- dvcs - isolation done with distinct repositories
- in the book it looks like this is most common way to do branch development
hg branch , a.k.a. named branches :
- standard builtin command(s)
- enable in-repo branches
- can be transfered accross the wire
- the subject of this article
- reference here
bookmarks extensions
- hg bookmarks - standard mercurial extension
- needs to be enabled
- transfered if both sides enable extension
- IMHO very similar to named branches but not so elegant
- reference here
anonymous branch
- hg up -C + hg ci -m”commit on old revision”
- system similar to named branches, but with no names ;)
- reference here
mq extension
- extension for management of mutable changesets, that can be used for branch development too, but haven’t yet figured how ;)
- seems a bit complicated for my needs
- reference here

Between all these options I chose named branches as the best way to do in-repo branches (simple, native, handy, works accross the wire, just works). The basic workflow is explained in next sections.

Commands

I won’t explain much, I’ll just list operations used in standard workflow of branch development.

Create new named branch

The command will create new branch named for instance, after feature nr. 91 (such branch should not previously exist). After command finishes, it will switch working copy to newly created branch:

<default> hg branch feat91
<feat91>

List all created branches

<> hg branches
default
feat91

On which branch am I?

<feat91> hg branch
feat91

Switch working copy to branch

The command will switch working copy (with local changes) to branch feat91:

<default> hg update feat91
<feat91>

(some documentation suggests using -C option but it works without -C).

First push of newly created branch to other repo

First push of newly created branch to other repo needs extra option —new-branch:

<> hg push --new-branch

Next pushes goes normal (hg push).

See incomming changes from feat91 to current

How to see incomming changes from feat91 to current - default branch (preview merge):

<default> hg merge -P feat91

Merge changes from branch feat91 to default

<default> hg merge feat91

check changes and commit:

<default> hg commit -m"merged feat91 -> default"

Merge changes from default branch to feat91

<feat91> hg merge feat91

check changes and commit:

<feat91> hg commit -m"merged default -> feat91"

Close a branch

There are many ways to do it, but this one I found as the most regular one. Workflow:

list branches

<feat91> hg branches
default
feat91

close a branch:

<feat91> hg commit --close-branch -m "close feat91"

The command will just chang status of branch to “closed”, the branch won’t be stripped, and you can come back to it anytime, and if you like you can reactivate it and continue development on it.

After closing it is advised to switch to an active branch:

<feat91> hg up default

Default listing of branches won’t show closed:

<default>hg branches
default

But if you like you can list closed too:

<default> hg branches -c
default
feat91 (closed)

Like noted before, after closing you can switch to closed branch:

<default> hg up feat91
<feat91>

You can make changes and commit - what will automatically put branch to “reopened” state:

<feat91> hg ci -m"commit in feat91 after close"
reopening closed branch head 11

List branches after reopening: <> hg branches -c default feat91 (inactive)

and you can close it again:

<feat91> hg ci --close-branch -m"closing feat91 again"

View branches log graphically

hg serve won’t show this graphically, but there is extension graphlog which can show this in console. http://mercurial.selenic.com/wiki/GraphlogExtension Enable it in .hg\hgrc

[extensions]
hgext.graphlog =

and try:

<default> hg glog
...

| o  changeset:   8:4b4d4b76110b
| |  branch:      feat91
| |  user:        trebor74hr
| |  date:        Sat Sep 03 18:47:43 2011 +0200
| |  summary:     new feature in feat91
| |
| o  changeset:   7:8d78dd2d7334
|/|  branch:      feat91
| |  parent:      4:0d29e1ecb689
| |  parent:      6:04ec650ace3c
| |  user:        trebor74hr
| |  date:        Sat Sep 03 18:45:04 2011 +0200
| |  summary:     merged default -> feat91
| |
o |  changeset:   6:04ec650ace3c
|\|  parent:      5:3c4a79f2ac3b
| |  parent:      4:0d29e1ecb689
| |  user:        trebor74hr@gmail.com
| |  date:        Sat Sep 03 18:39:28 2011 +0200
| |  summary:     merged feat91 -> default
| |
o |  changeset:   5:3c4a79f2ac3b
| |  parent:      2:fbc6ab30159e
| |  user:        trebor74hr@gmail.com
| |  date:        Sat Sep 03 18:34:06 2011 +0200
| |  summary:     ovaj ci u default branch-u zelim u feat91 branch prebaciti
| |
| o  changeset:   4:0d29e1ecb689
| |  branch:      feat91
| |  user:        trebor74hr
| |  date:        Sat Sep 03 18:33:54 2011 +0200
| |  summary:     ovo je f1 commit u feat91
| |
| o  changeset:   3:308b5d06a330
|/   branch:      feat91
|    user:        trebor74hr
|    date:        Sat Sep 03 18:29:52 2011 +0200
|    summary:     branch feat91 commit
|
o  changeset:   2:fbc6ab30159e
|  user:        trebor74hr@gmail.com
|  date:        Sat Sep 03 18:28:07 2011 +0200
|  summary:     noc
...

Other references

A guide to branching in Mercurial with comparison to git - a bit obsolete but good
Managing releases and branchy development
Mercurial wiki: BranchingExplained - with comparision to git, a bit obsolete, doesn’t cover named branches

How to count characters, bytes and words in Vim

Sun, 09 Jan 2011 01:06:00 +0100

The article provides some information and tips on the subject: how to count characters, bytes and words in Vim.

Vim status line

When you do selection in vim in the status line there is information on how many chars/lines are selected:

when selection is in one line then you can see nr. of chars:
```
-- visual --                                    34
```
when selection goes over several lines then you can see nr. of lines:
```
-- visual --                                     2
```

Get me more information

I wanted to get information on how many chars are selected in selection over several lines. When I do selection and press the sequence (normal mode):

g+(CTRL+G)

reference :help v_g_CTRL-G

that gives me the information:

when there is no selection:

col 34 of 60; line 193 of 216; word 927 of 1046; byte 7216 of 8226

when you select something:

selected 1 of 216 lines; 4 of 1046 words; 45 of 8226 bytes

I want to count something special

Vim version 7 - adds option [n] to :s(ubstite) command:

[n] Report the number of matches, do not actually substitute. The [c] flag is ignored. The matches are reported as if ‘report’ is zero. Useful to |count-items|.

reference :help s_flags

This option can produce more power count functions, i.e. you can count number of matches of any regular expression. Examples:

:%s/.//gn - count all chars

5544 matches on 176 lines

:%s/\S//gn - count non-space chars

4080 matches on 167 lines

:%s/\s//gn - count space chars

1413 on 148 lines

Note

More information how to count items in vim can be found:

:help count-items

There you can find examples to count with :s command with [n] option, but there are mostly in the form:

   :%s/./&/gn

i.e. they use & as replacement char (replace the same). Since option [n] doesn’t replace anything the effect is the same as:

   :%s/.//gn

I tested in Vim version 7 and the behaviour is like described.

Go on windows tutorial (#golang)

Wed, 05 Jan 2011 00:15:00 +0100

Recently I invested some time to learn about new and popular language Go. I wanna give a test-drive, and since I have Windows XP and there is no GO official windows Go port, I needed to do some manual work.

This is the procedure:

download go windows port (mingw) from http://code.google.com/p/gomingw/downloads/detail?name=gowin32_2010-12-22.zip&can=2&q=
unzip to some folder. I did this in c:\testing\go, so I have c:\testing\go\bin\8g.exe
create batch file set-go.bat and put it in the folder that is in sys path, e.g. c:\windows\system32\set-go.bat:

   @echo off

   set GOROOT=c:\testing\go

   set GOOS=windows

   set GOARCH=386

   SET PATH=%GOROOT%\bin;%PATH%

   echo Set GOROOT=%GOROOT%

Start GO env. shell:

start DOS prompt, e.g. start/run/cmd
call set-go

Someone could ask why I didn’t put these vars to global system path (my computer right click, props, env. vars …). The answer is: I don’t do things like this, I like having more control over my shell environment. If you want to set it globally - check this link.

Now environment is ready, and I can write go programs, compile and run them like this:

create new file hello.go

   package main

   func main() {

   println(“Hello world.”)

   }

go to the folder in your DOS-GO prompt and compile it - output is in hello.8:

8g hello.go

link it - output is in hello.exe

8l -o hello.exe hello.8

run it:

> hello.exe

Hello world.

Additional notes:

Go dev team is developing official windows port - so stay tuned
On http://stackoverflow.com/questions/1717652/can-go-compiler-be-installed-on-windows you can find other useful information and similar tutorial.

Good luck Go-ing!

Tubmlr per-tag RSS feed

Sun, 24 Oct 2010 00:30:00 +0200

Tubmlr supports tagging. This post is tagged with “tumblr”. The page that lists all posts tagged with “tubmlr” is:

http://trebor74hr.tumblr.com/tagged/tumblr/

Although the page provides RSS feed:

http://trebor74hr.tumblr.com/rss

this is not per-tag feed, you RSS feed for all posts :(.

Solution

Hopefully there is solution, RSS feed that provides only posts tagged with tag “tumblr” is:

http://trebor74hr.tumblr.com/tagged/tumblr/rss

And it works !!!