Saturday, June 30, 2007

If you find you lack the ability to sudo...

While trying to setup Subversion, I needed to add myself to the subversion group. What I ran was "sudo usermod -G subversion sam". That was bad. Not, oh there is no more milk kind of bad. Bad like you just ran over a family member sort of bad. A section of the usermod man page will explain:

-G groups Supplementary groups given by name or number in a comma-separated list with no whitespace. The user will be removed from any groups to which they currently belong that are not included in groups.

Notice that second sentence? I didn't. I continued on my merry way until I needed to sudo. I ran sudo, and nothing happened. No error. Nothing. Being the only user on the system, I frantically tried to reverse the situation, since I would be able to do little of import without this. In the end, I found a related tutorial that let me fix it. The steps:
  • Restart the computer, and be at the console.
  • Press Esc when GRUB comes up.
  • Hover over the latest kernel (what you would normally boot) and press e (to edit it).
  • Go down to the line starting with "kernel /boot/vmlinuz" and press e.
  • At the end of that line, press space and then type "single".
  • Press b to boot into that kernel.
  • Once the system loads, you will be in single user mode, as root.
Of course, as root, you can change your password, your normal user's password, or (as in my case), add yourself back to the admin group.

After you login, you will be able to sudo, but you may notice some other tasks not working. In my case, I was unable to play any music or video files. I was baffled. I checked the permissions, they seemed fine. It turns out there are a lot of groups your user is put in by default, e.g. audio, cdrom, etc, which let you do things like that. If you use Gnome, go to System, Administration, Users and Groups. Find your user, view Properties, and then User Privileges. Check all those boxes, and all should be fine. Alternatively, you could make a new user and run id on it, comparing their groups to yours. This would work on default installs, without any custom user creation scripts.

As a side note, this method creates an extreme security flaw in certain systems. It is possible to turn off interactive GRUB editing if desired. For personal systems without sensitive data, it is best to leave it on, for eventualities like the one I encountered.

Personal Settings in SVN Increases Happiness by 125%!

This may sound a little silly, but when I first started using aliases in Linux (and loving their potential utility), I was put off by the realization that once I got used to using them, I would end up being on another machine without them. This annoyance made me resistant to using them for some time. Eventually, I started using them so much, that I copied my .bashrc (and .vimrc and other such files) to all the remote servers I would need to interact with regularly.

This helped quite a bit, but it introduced another level of annoyance. Whenever I made a change, improved a command, added a new alias, I would have to add that to all the copies I had floating around. Dumb. Dumb and annoying.

So, I thought, why not set up a repo? While I use SVN regularly, I had never setup one on my own before, so it seemed like a win/win situation. In sum, the following is what I did to get it all to work.

The first thing to decide is how the repository will be accessed, either via SSH or Apache. The use of Apache is very common, and there are many tutorials on how to do it (like this one). I chose not to use Apache for 2 reasons: I did not want to have to be concerned with securing Apache on the server in question unless necessary, and secondly any computer on which I would want to access the repositories would have SSH. If you want to be able to see your repo in a browser, use Apache (or Trac).

Initial steps:
sudo apt-get install subversion # As of now on Feisty, this installs 1.4.3
sudo mkdir /home/svn/configfiles #For the repo, easy place to remember

You can replace "configfiles" with whatever you want to call the repo. For this case, it does not matter a lot, since I will be checking out the files in many various places, not all in one directory.

Now for permissions and such:
sudo addgroup subversion
sudo usermod -a -G subversion YOURUSER

And the real magic:
sudo svnadmin create /home/svn/configfiles/
sudo chown -R subversion configfiles # make sure perms are right after repo creation
sudo chmod g+rws configfiles # Ditto
sudo svn import /your/initial/files file:///home/svn/configfiles/repo -m "initial import"
sudo apt-get install xinetd

Now run
sudo vim /etc/xinetd.conf
and add:

service svn
port = 3690
socket_type = stream
protocol = tcp
wait = no
user = www-data
server = /usr/bin/svnserve
server_args = -i -r /home/svn

Now go into the conf dir in your repo and
sudo vim svnserve.conf
and edit what you like. I uncommented "anon-access = read" and "password-db = passwd".

You are done! The test: "svn co svn+ssh://SERVERWITHREPO/home/svn/configfiles". You will be prompted for as password 3 times, do not worry. If all is well, you should see "Checked out revision 1.", and the file(s) in the repo will not be in the dir where you ran the above command.

For more info, see the SVN Book. Thanks to this guide for getting me through the main steps.

[NOTE:] After some experimenting, I ended up creating the repo like this:
sudo svnadmin create /home/svn/configfiles/trunk
sudo svn import ~/.bashrc file:///home/svn/configfiles/trunk/bashrc -m "Initial import of .bashrc"

Then I checked it out via:
svn co svn+ssh://SERVERWITHREPO/home/svn/configfiles/trunk ~/DIR/TO/CHECKOUT/IN

Also, the way to delete a repo completely (as I did during testing) is simply to remove the entire dir (e.g. "sudo rm -rf /home/svn").

Friday, June 29, 2007

Exporting a MySQL DB to .csv

I thought the operation would be simple. Short, no casualties, 100% chance of success. No such luck.

I needed to export the data of a MySQL database into a .csv format file. I did not want it in a huge XML file, and I did not need the actual .sql statements. A common need, one would think. The command to use is:
mysqldump -u root --fields-terminated-by=, --tab=/tmp --tables DATABASE

Now, this might seem simple. But what I tried first was having the " > dump.sql" file at the end, and had the path after "--tab" as my home directory. Now the first choice was fine, as you can separate where you send the data files versus where you send the table syntax. But, you will get an error:
mysqldump: Got error: 1: Can't create/write to file 'dumpdir/tablename.txt' (Errcode: 13) when executing 'SELECT INTO OUTFILE'
A post on the MySQL forums clued me in:
It's because the mysql user doesn't have the right permissions to write to that dir.
a simple chmod should fix the problem

So that led me to make a dir and chown it to the mysql user, which worked. However, if you write it to /tmp, you do not need to chown. So this version writes all the files to one dir without permission woes:
mysqldump -u root --fields-terminated-by=, --tab=/tmp --tables DATABASE

New and Helpful Aliases

Some aliases I have added of late and found to be useful:

Displays a calendar and then your locale's 12-hour clock time:
alias now='cal;date +%r'

For those who often like to edit X display config:
alias editxorg='sudo vim /etc/X11/xorg.conf'

For those use window managers without a shutdown button:
alias turnoff='sudo shutdown -h now'

Does all you need to do for updating:
alias updateme='sudo aptitude update && sudo aptitude upgrade && sudo aptitude
dist-upgrade && sudo aptitude autoclean'

I am also working on something that I hope to works as follows. "help CMD" will run "CMD -h" or "--help", checking the results. If that fails, it runs "info CMD", then "man CMD". The idea being that you can use one command to display information on a command, and it will get you to the first that has something to show. This would be helpful when one runs into programs that do not handle -h, or only have a man page, etc.

Summary Execution

I find myself coming across strange files on occasion. Crowded folders, shady thrice nested system dirs and the like are filled with them. Usually system files, files whose nature and purpose I might not know. To get to know such individuals, there are several built-in tools I might employ. I could run file on them, cat them, tail them, head them, run ls -lh in the folder, et al. But this takes too many steps. I want one command. And if I can get what I want in 5 steps, I can get it in one. It is the Linux way.

Thus I created the "summarize" function. It's a simple script that basically just grabs data using the methods I listed above, and formats them into an easy-to-read summary. Right now, it consists of:
# A command to provide lots of
# info about a file at glance!
FILE=`file $1`
echo "$FILE has `wc -l $1 | cut -d" " -f1` lines." &&
echo -e "\e[1;34mOwned by: \e[0m`ls -l $1 | cut -d" " -f3`" &&
echo -e "\e[1;34mGroup: \e[0m`ls -l $1 | cut -d" " -f4`" &&
echo -e "\e[1;34mSize: \e[0m`ls -lh $1 | cut -d" " -f5`" &&
echo "**********************************"
echo -e "\e[1;34mTop bits: \n \e[0m" &&
head -n 5 $1 &&
echo "" &&
echo -e "\e[1;34mBottom bits: \n \e[0m" &&
tail -n 5 $1
exit 0
One handy thing I learned while writing this was how to display text in chosen colors in bash, using the "echo -e" syntax. I find this makes it easier for my eye to quickly move to different fields.

You can also add something like this to your .bashrc:
alias summarize='/location/of/script/'

Example output on a test .txt file (screenshot to show colors. Your base text color will be different):

[EDIT, 07/03/07]: It is also useful to add contextual lines to the head and tail output. This can be done via:

head -n 5 $1 | nl -b a
nl -b a $1 | tail -n 5
This prints the first and last five lines, numbered according to the number of lines in the entire file. I find this makes it much easier to immediately see how long the file is. (Additional note: without the "-b a" option, nl will print lines for each textual line, skipping blank ones. I myself would rather the blank lines also be included, as this is standard practice.)