aasm_event :do_life

Over the past week after work, I've been working on a little side project to hone my ruby scripting skills and I'm ready to release it today.

I've written a script in ruby to parse through all the parliamentary reports from http://www.parliament.gov.sg/Publications/votes_11thParl.htm (at of time of processing 27th April 2010 report is the latest) and extract out the name, ward and date of absentee from each individual report and dump it into a SQLite database.

Some interesting facts:

1) Since the 11th Parliament started till the sitting on the 27th April 2010, there has been 928 absentee records.

2) The top 5 absentees in Parliament are:

Mr Lee Kuan Yew - 46
Dr Balaji Sadasivan - 38
Prof. Thio Li-ann - 38
Dr Loo Choon Yong - 28
Mr George Yong-boon Yeo - 25

3) Mr Low Thia Khiang (Hougang - Workers Party) have not been absent since the 11th Parliament commenced. Mr Chiam See Tong (Potong Pasir - Singapore People's Party) 13 times. Ms Sylvia Lim (NCMP - Workers Party) 6 times.

4) The top 5 wards which has the highest number of absentees in Parliament are:

Nominated Member - 146
Tanjong Pagar - 90
Pasir Ris-punggol - 75
Ang Mo Kio - 69
Aljunied - 62
West Coast - 56

5) The top 5 Parliament sittings with the most number of absentees are:

12th April 2007 - 28 (33.333% of Parliament)
24th March 2009 - 24
19th August 2007 - 23
20th August 2007 - 23
22nd Jan 2008 - 22

I'm releasing the database under the Creative Commons 3.0 license, so you are free to use the data for any purpose (analysis/data visualization/etc.) without any restriction :) Although I'd love it if you would share with me where are you using the data :)

Grab it in either formats:

SQLite

CSV

Update: Here's my ruby script for those who are curious. (If you run this on your own, you might notice that RAdm Lui's records are slightly misnamed. I had to manually edit those records to clean up the name for the released database. If you can improve on the script, I'll appreciate it too)

Disclaimer: Due to the nature of a computerized and automatic processing, there may be some inaccurancies in the data collected, although as far as I know it, the initial parsing doesn't seem to output any errors/discrepancy. If you've found something wrong, please contact me so that I can fix it.

Some thoughts from YouthQuake 7 - CAN YOUNG SINGAPOREANS AFFORD PUBLIC HOUSING IN SINGAPORE TODAY?

After spending a couple of hours trying to get integrity setup, I thought I'll
share and document my process for future reference.

So what is Integrity?

From their website:

"As soon as you push your commits, Integrity builds your code,
run your tests and makes sure everything works fine. It then reports
the build status using various notifiers back to you and your team so
everyone is on the same page and problems can be fixed right
away."

In a nutshell, Integrity is a Continous Integration Server which helps to ensure that your codebase is
properly integrated and working at every stage of your development.

Integrity's website> is pretty
straight-forward and simple to understand, head on over to their
webpage at http://integrityapp.com and follow the instructions on the
page. The bottom part of the page is pretty confusing so what I
basically did was to:

 
$ gem install bundler 
$ git clone git://github.com/integrity/integrity 
$ cd integrity 
$ git checkout -b deploy v0.4.1 
$ bundle install 
$ bundle lock 
$ rake db 

Open up the default init.rb and add in the following lines in
between the Integrity.configure block

 
c.user "username" 
c.pass "password" 

What this does is tell Integrity to set a HTTP Authentication check
for the application so that only people with the proper credentials
can access your application.

So here's where the confusion starts. The tutorial over at
Integrityapp.com makes no mention about how to get your database.yml
file copied into the builds/ directory. If you're like most developers
(and I hope you are), your database.yml file will not be injected into
your repository, so how do we setup Integrity such that it would be
able to copy the proper .yml file and configure it according to your
settings?

First, setup a rake task that does the magic of copying. In your
/lib/tasks create a .rake file and copy the following:

 
namespace :test do 
 task :move_test_db_config do 
   require 'ftools' 
   File.move(RAILS_ROOT+"/config/database.yml.test", 
RAILS_ROOT+"/config/database.yml") 
 end 
end 

This rake task will copy the file database.yml.test to database.yml,
so you're going to need to create a database.yml.test file in your
/config directory.

You don't have to put in other information like your
development/production settings since all integrity needs is your test
environment.

Here's mine:

test: 
 adapter: sqlite3 
 database: db/test.sqlite3 
 pool: 5 
 timeout: 5000 

Yeah, dead simple.

With that said and done, all you need to do now is to start the app,
access the app, and create a new project.

In the 'build script' field, replace it with "rake
test:move_test_db_config" and you're done!

As for notifications, just follow the instructions at Integrity :)

protip: Make sure you give the user that is running the webserver
(apache/thin) access to the remote git repository that you want to
track. Use SSH key authentication for best results.

Source:
[1] http://integrityapp.com
[2] http://locomotivation.squeejee.com/post/109227509/ruby-on-rails-integration-t...

Here's a useful tip for those who wants to be updated everytime someone pushes code into a shared git repository.

There's a hook that comes with git that sends out an email to a defined e-mail address with information about a commit that has recently been pushed. It tracks the author, the commit message, the commit log as well as displays the changes made.

To get started,

1) In your /remote_git_repo_path/.git/hooks (depending on how you configure your repo, you may not have a '.git' directory),

 mv post-receive-sample post-receive chmod +x post-receive 

 #!/bin/sh 


# An example hook script for the "post-receive" event. 
# "post-receive" script is run after receive-pack has accepted a pack
# and the repository has been updated.  It is passed arguments in through 
# stdin in the form 
# For example: 
#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master 
# see contrib/hooks/ for an sample, or uncomment the next line and # rename the file to "post-receive".
#. /usr/share/doc/git-core/contrib/hooks/post-receive-email 

3) Ensure that you have the file /usr/share/doc/git-core/contrib/hooks/post-receive-email. If you do, uncomment the line in your post-receive file

 #. /usr/share/doc/git-core/contrib/hooks/post-receive-email . /usr/share/doc/git-core/contrib/hooks/post-receive-email 

4) If you don't have it, don't fret, copy the contents from here into a readable directory and call the file 'post-receive-email'. Point to that file with its full path in your post-receive script.

5) Once that done, all you need is to configure the repository to send the email to the correct address. Still in your remote repository, edit the .git/config file and add the following:

[hooks]
mailinglist = whatever@email_address_here
showrev = "git show -C %s; echo" 

git show -C %s;echo 
# will ensure that the full git diff will be embedded in the e-mail 

And you're done! Now in your local repository, make your changes and push it up. You should receive an e-mail in your configured e-mail address.

Extra Tip - To get rid of the 'UNNAMED PROJECT' line appearing in your email subject line, in your remote repository edit .git/description with whatever you want to appear.

Hope this helps :)

Finally decided to upload my vimrc, gitconfig and bashrc config files onto a repository on github.

If you guys got any extra tips or must-have configs, please share :)

That incident made me realize that I needed to enforce a stricter git workflow to ensure things like this doesn't happen again. After talking to the guys over at #git (irc.freenode.net), most recommentation was to activate the update git-hook which was supposed to check if the code was force-pushed before accepting it into the repository. I've since added that hook but I'm curious, how are you guys managing your git repository and enforcing your workflows?

Just a couple of reminders for myself, in case I spend wasted time trying to get gmail smtp working on actionmailer again.

To get your ActionMailer configured for gmail:

# in config/environments.rb 
 # add the following lines AFTER your Rails::Initializer.run block

  email_settings = YAML::load(File.open("#{RAILS_ROOT}/config/email.yml"))

  ActionMailer::Base.raise_delivery_errors = true
  ActionMailer::Base.perform_deliveries = true
   ActionMailer::Base.delivery_method = :smtp
  ActionMailer::Base.smtp_settings = email_settings[RAILS_ENV] unless email_settings[RAILS_ENV].nil?

# in config/email.yml (if its not there, create it)
 
# add the following for each rails environment

development:
  :enable_starttls_auto: true
  :address: smtp.gmail.com
  :port: 587
  :authentication: :plain
  :user_name: <whatever@domain.com>
   :password: <password>
  :domain: gmail.com

# remember the extra ':' before the yaml key. This is what wasted my time

And magic.

There's no need to install any new smtp/tls libraries as there's already built in support for tls authentication within ActionMailer. Just make sure to set 'enable_starttls_auto' to true.

(a & b).any? or (a & b).empty?

* 'or' in the snipplet above, is not an operator, but a geniune 'or'

From The Momoro Hoax