Tuesday, July 27, 2010

Response to .DOC Attachment

What do you do when you receive an email file attachment in Microsoft's .DOC format? Sure, we could open it in OpenOffice.org and go about our business, but there's no guarantee that this will remain true for future versions of .DOC. Use of proprietary formats are a danger to interoperability and to future innovation. One good thing to do is to reply to the email and explain why they should send the attachment in a standard format. I've seen several canned email responses in the past, but most are too brief to explain the issue, or too harsh in their wording to satisfy my tastes. I was prompted to write a response after receiving such an email attachment today, and I have combined some of the best ideas from elsewhere and added some of my own. Here is what I would recommend:
The document you have sent was not saved in an accepted format for Internet mail.

It was saved in a proprietary format that is unreadable on several types of computers because the method for decoding the document is kept secret by Microsoft and is purposefully changed with each new release of Microsoft Word so that existing users of Microsoft Word will be forced to pay for expensive upgrades in order to continue to read Word Documents sent by others. For example, in 2010, Microsoft Office Home & Business 2010 was priced at $279.99, and Microsoft Office Professional 2010 was priced at $499.99.

Recent versions of Word have started using a newer, patented OOXML format. In many countries, it is actually illegal for other products to decipher this format. This is a lock-in technique used by Microsoft to maintain their monopoly on Office software, and by extension, their monopoly on the operating system market, since they have not released a cross-platform version of Microsoft Office compatible with other operating systems.

In most cases, the size of the file saved in Microsoft's secret, proprietary format is also substantially larger than a standards compliant file containing the same information and the same formatting.

It is also important to note that Microsoft Word documents are often infected with viruses. Excel, Access, and Power Point files are also vulnerable to infection. This potential for infection is largely due to the Macro language and the "Visual Basic for Applications" language which are built into the format to provide powerful programming capabilities. While powerful, these features were not protected with proper security precautions, and the majority of users do not actually use these features or even know that they exist.

What to do instead:

If you continue to use Microsoft Word, please have the courtesy to “Save As” one of the following formats: ODT (if available), DOS Text, HTML, or Portable Document Format (PDF) and after saving, send the resulting file as an attachment.

Alternatively, you could use a product such as AbiWord, KOffice, Google Docs, NeoOffice, or OpenOffice.org that allows you to save your document in the Open Document Text Format (ODT), which is an ISO/IEC International Standard, and is supported by such notable companies as Apple, Adobe, Google, IBM, Intel, Nokia, Novell, and Sun Microsystems. If you don't have one of these programs, I would recommend OpenOffice.org, which may be downloaded free of charge and used for any purpose, personal or commercial.

A third option is to simply type your message directly into mail (instead of typing into Microsoft Outlook or Microsoft Word) so that you won't need to use an attachment at all.

In the highly unlikely event that your document cannot be converted to an open, non-proprietary format, consider printing it and mailing it by post, or scanning it, and sending it in a standard graphic format such as PNG or JPG.

Thank you.

Tuesday, July 13, 2010

Open Database Conventions

I've been using a set of database table and field naming conventions and related standards that have been gradually adopted and refined over the last 10 years. I decided today that it would be a good idea to share these with others, particularly because I decided to bring Jim from UVOG in on a database project I'm starting on. I realized that sharing them with others on an even wider scale might be a good thing on the off-chance that, if someone else out there adopts them, and if our code ever ends up crossing paths, we will gain greater interoperability and understanding on account of using the same techniques.

Feel free to take some or all of these ideas and implement them in your own projects. I am willing to hear feedback and incorporate it back into the standard if you think there's a better way to do something that I've mentioned.

Some of these conventions will apply specifically to MySQL, while others will apply to the greater spectrum of databases. My experience spans MS Access, MS SQL Server, dBase, FoxPro, the Apollo Database Engine, and MySQL, with some very light experience in SQLite and PostgreSQL, but the bulk of my work today happens in MySQL.

Naming the Database

The name of the database itself is the least of my concerns. I would just recommend that in code that operates with or upon the database, this value should be easily configurable along with the hostname, port, and authentication credentials, so that any database could be selected.

Naming Tables

The name of a table should be a succinct name descriptive of what is to be represented by a single record in the table. If each record in the table represents an account, for example, name the table "account" (in the singular, all lowercase.) The reason we use singular forms is so that dot notation naming of fields makes sense when read out loud, for example, account.balance. There are many reasons we use all lowercase. Most production Database servers are case sensitive, and by using all lowercase we can eliminate the possibility of an error relating to case. It also improves code readability when SQL statements are written out with keywords and function names capitalized and with table and field names in lowercase.

There is a special case of naming in the event of a junction table. (A junction table is used to create a many-to-many relationship between two tables, and consists of nothing more than an id field, and foreign keys for the two tables whose records it is tying together.) The name of a junction table should be the name of the other two tables mashed together, with the name of the greater entity first. For example, if I have a company table and an customer table, and customers are potentially associated with multiple companies, I would name the junction table companyemployee, and it would contain id, kcompany, and kemployee, in that order. If there is a toss-up in determining which entity is "greater," place the two table names in alphabetical order.

Naming Fields

Always name your fields as though they are going to be used in dot notation. We want account.balance, not account.accountbalance, so do not redundantly repeat the table name within the field name. Avoid abbreviations that seem like a hack. For example, number is better than "no" or "num." Widely used abbreviations like ipaddress, ssn, or id are ok--be particularly careful to avoid expanding acronyms that are not expanded in common speech.

The Primary Key

In order to be properly maintained, every record needs to be addressable by a unique identifier. This identifier should not be meaningful in any real-world way for reasons that are explained in depth elsewhere, but I will give one brief example. If you were making an employee database, you might be tempted to use a person's SSN as the primary key for the employee table. Three years after the program is written, your company might hire someone who is in the country on a work visa, and therefore does not have an SSN, and now your system is broken. One will follow this line of thinking out and conclude that the key should be completely arbitrary and meaningless so that no such conflicting situation may ever occur. Additionally, for the sake of consistency, the primary key field in each table you make should be of the same name and type. I recommend naming the first field of each table "id", setting it to "auto_increment," and setting it to become the PRIMARY KEY index.

The value of this id field should be used internally for programming purposes and for describing relations between tables, but it should never be shown prominently to the user. If it is shown, it is merely for debugging or reference purposes. The number in this field, once generated, should not be changeable by the user, is not guaranteed to be sequential with the records around it, and is not guaranteed to fall into any range of values more specific than those allowed by an int (11) field.

Foreign Keys

A foreign key can be thought of as a "pointer" referring from the current record out to another record in another table (or possibly another record in the same table.) All foreign keys should be prefixed with the letter "k" (meaning key), and should either be named "parent" if the key is referring back to the same table for the purpose of producing a hierarchical system, or if referring to another table, it should be given the name of the table into which it points. If more than one key in a record must point to the same table for different purposes, the purpose should be appended to the name followed by an underscore. For example, kaddress_from and kaddress_to would be pointers to two records in the table named "address." The relation to one of them is described as "from" and the other as "to." This is the only time an underscore is permitted to be used in a field name, and the purpose of the underscore is to make the table name unambiguous.

When possible, try to put keys to parent records or records which are considered to "own" the current record near the top of the field listing, following just after id.

If these standards are followed for the primary key and foreign key fields, I could look up any reference using the following technique:

Given a field whose name begins with "k": Take everything in the field name following the "k" up to the end of the name, or up to (but not including) the first encountered underscore "_" character, and consider this to be the "lookup table name," if such a table exists. Retrieve the data by issuing a query in the form of:

SELECT * FROM [lookup table name] WHERE id = [value of the field]

Choosing Appropriate Data Types for Fields

Always choose the most restrictive type that can safely store the data, including valid values that you might not anticipate.


For id fields or foreign key fields, use INT (11)

For boolean (yes/no or true/false) fields, use TINYINT (1) A value of zero means false, and any nonzero value means true. You can distinguish between individual nonzero values for record housekeeping if you wish, as long as others reading the data as merely true or false will not be under a misconception from not making such a distinction.


For currency values less than a million dollars, use DECIMAL (8, 2) --- increase the first size by a sufficient number of digits if you need larger currency values.


For numeric data involving whole numbers or integers only, use INT (11) --- unless the size is astronomical, in which case, you will need to upgrade to a larger type.

For weights or non-integer quantities less than ten thousand, use DECIMAL (8, 4) --- increase the first size if values beyond ten thousand are needed, or both sizes if more precision than four decimal positions is needed. Four decimal places was chosen so that a number like 12.05% could be fully represented as 0.1205.


For date or time values or stamps where the "time part" has meaning (particularly in sorting), use the DATETIME type. A value of "0000-00-00 00:00:00" means not entered or unknown.

For date values where the time is irrelevant, use the DATE type. A value of "0000-00-00" means not entered or unknown.


For multiple choice values: If there are a discrete number of choices presently and in the near future, few choices (about 8 or less), and little or no benefit would be had by allowing the user to customize the set of choices, an enumerated value may be used. Example: alignment ENUM ('left', 'right', 'center') DEFAULT 'left'

For multiple choice values where the choices are numerous or user defined or potentially user managed, a foreign key and a lookup table should be used instead of an enum.


For any string data: If the options are very well established, choose the smallest size category from the table below that can hold all possible values, or for data that is free-form, choose one category larger than you think is really necessary. I will give some example with each size category:

VARCHAR (4) -- A standard name suffix like Jr., Sr., II., III.
VARCHAR (8) -- Data known for certain to be less than or equal to 8 characters.
VARCHAR (16) -- A zip code. (Minimum to store a US zip code is presently 9 digits plus one hyphen.)
VARCHAR (24) -- A phone number, with decorations. The following number is 23 characters long, including spaces: +1 (541) 375-0448 x8888
VARCHAR (32) -- Product codes, SKU numbers, or generated numbers that are guaranteed to be less than or equal to 32 characters.
VARCHAR (48) -- A first(given) name by itself, or a last name(surname) by itself.
VARCHAR (64) -- A full name field (first and last name together.)
VARCHAR (128) -- An email address.
VARCHAR (248) -- A web address.


For anything multi-line, or potentially longer than 248 characters, use the TEXT type. If the data or text will be more than a couple of kilobytes, investigate the LONGTEXT type.


I don't like big files taking up space in my InnoDB tables and slowing down my replication server. Instead, I store the original filename only (and only as a convenience) in a VARCHAR field, and then save the file separately named based upon the table name and record number the file is associated with. If I need to replicate these files, I let rsync take care of that. If the "attached" files MUST be synchronized with the other data in the record at any given moment, then it might be acceptable to resort to BLOB fields.

Order of Fields

When creating tables, the order of the fields really doesn't matter to the database engine. The sequence, therefore, should be chosen for purposes of clarity in documentation.

Well, that's all I have for now. Have at it, tear it up, and send me suggestions.

ACNG Client side perl script part two

Please refer to these previous articles : Setting up ACNG ACNG client side perl script for set up and scripting Apt-Cacher-NG.

I have made some modifications to the perl script to allow for simple gui integration.
Props to @gpled for showing me zenity a couple of months ago.

Change this:
#!usr/bin/perl -w
#script by @robots_unix

print "Are you at CCC?";
$name = ;
chomp ($name);
if ($name eq "yes"){

To this:

#!usr/bin/perl -w
#script by @robots_unix
system('zenity --title "Apt-Cacher-NG" --text "Are you at CCC?" --list --radiolist --column "ACNG" --column "Toggle" False no True yes> aptyesno');

open(APT, "Location of aptyesno");
$name = ;
if ($name eq "yes\n"){

Then right-click on gnome-panel, click add to panel, click create custom application launcher, then for the command type: sudo perl location of your perl script.

Now you can change the hosts file by just clicking an icon on your gnome-panel.

Have fun.

Monday, July 12, 2010

Moving the close, minimize and maximize to the right side again

I just installed Wubi and Ubuntu 10.04 and love it!!
But I was used to the close buttons being on the right

type in the terminal
gconftool-2 --set /apps/metacity/general/button_layout --type string menu:minimize,maximize,close

you can also change it manually:

just type in terminal:

then open apps -> metacity -> general
and change button layout on the right side to:

I hope that is helpful

Linux Chrome fix backspace as back button

I like to be able to go back when browsing without using my mouse. I have found an extension which fixes the backspace so that it now goes back.

Install the extension:

Then click the radio button:
Activate Backspace for navigation in history object

And hit save. That is all.

Speeding up your Updates with Ubuntu and APT Cacher NG

If you are in a building with more than one computer running Ubuntu, you may have wondered why everyone has to download updates separately. The answer is, they don't. Here's how to accomplish it with a tool called apt-cacher-ng.

apt-cacher-ng is a fork of a project called apt-cacher, which in turn is an alternative to apt-proxy which is poorly maintained and upon my testing, was unreliable.

Setting up the APT Cacher Server

Begin by installing apt-cacher-ng from Synaptic Package Manager, or from the Terminal with the following command:

sudo apt-get install apt-cacher-ng

By default, the version of apt-cacher included with Ubuntu does not currently include security updates. We can easily add the security updates in, however, by following these steps:

As root, create a new file called /etc/apt-cacher-ng/ubuntu_security

sudo nano /etc/apt-cacher-ng/ubuntu-security

This file will be a list of mirrors from which the updates may be downloaded. We only want to insert a single line, the location of Ubuntu's official security updates server:


Save and close the file.

Next, edit the file /etc/apt-cacher-ng/acng.conf

sudo nano /etc/apt-cacher-ng/acng.conf

While we're here, you will see a couple of lines like this:

# Set to 9999 to emulate apt-proxy

I recommend that you follow the instructions and set the port to 9999 to emulate apt-proxy. Not only will this make your server compatible with systems expecting an apt-proxy server, but it will also make the port number easier for you to remember.

Next, look for this section:

# Repository remapping. See manual for details.
# In this example, backends file is generated during package installation.
Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian
Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu
Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol

We're going to add one more line at the end of this section to describe the Ubuntu Security repository that we are adding (make sure the following all goes on one line):

Remap-ubusec: file:ubuntu_security /ubuntu-security ; http://security.ubuntu.com/ubuntu

Now, save changes to this file and restart the apt-cacher-ng service.

sudo service apt-cacher-ng restart

If all went well, the server is now working. No you may proceed to setting up the clients. I strongly recommend making the server into a client of its own apt-cacher, as there is no reason for that system to download the updates from the Internet twice.

Setting up an APT Cacher Client

The basic idea of how to set up a client is to change all the lines in /etc/apt/sources.list to point at the local apt-cacher-ng server instead of at the internet servers. I recommend taking an additional step first so that you can easily flex between different servers. We will create a hostname alias called "apt-cacher" that points at your apt-cacher-ng server so you can simply re-point the hostname whenever you want to switch servers.

sudo nano /etc/hosts

We will add our own entry just after these two lines: localhost your-computer-name

If the apt-cacher-ng server is running on the same computer which you are setting up as a client, its IP address will be, otherwise, you need to find (or set) the static LAN IP address for your server. I will assume it is in this example because that's what it is in our building here at CCC. The line you will add will look like this: apt-cacher

Save and close the hosts file.

Now we will replace the entries in the sources.list to point at the cacher:

sudo nano /etc/apt/sources.list

The easiest way to handle this will be a search and replace.

First, replace every instance of "us.archive.ubuntu.com" with "apt-cacher:9999" (no quotes on either) If you are using nano, this is accomplished by pressing Ctrl+W, Ctrl+R, then entering the search string and pressing enter, then entering the string to replace it with, and pressing enter again, then when prompted, press "A" for all.

Next, replace every instance of "security.ubuntu.com/ubuntu" with "apt-cacher:9999/ubuntu-security" (no quotes on these, either.) If you are using nano, use the same steps as given above.

(In this example, we won't be handling the partner repository or any third party repositories which you might have installed.)

Save and close your sources.list. Your final sources.list if you are running Ubuntu 10.04 Lucid Lynx will look something like this:

# deb cdrom:[Ubuntu 10.04 LTS _Lucid Lynx_ - Release i386 (20100429)]/ lucid main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://apt-cacher:9999/ubuntu/ lucid main restricted
deb-src http://apt-cacher:9999/ubuntu/ lucid main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://apt-cacher:9999/ubuntu/ lucid-updates main restricted
deb-src http://apt-cacher:9999/ubuntu/ lucid-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://apt-cacher:9999/ubuntu/ lucid universe
deb-src http://apt-cacher:9999/ubuntu/ lucid universe
deb http://apt-cacher:9999/ubuntu/ lucid-updates universe
deb-src http://apt-cacher:9999/ubuntu/ lucid-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://apt-cacher:9999/ubuntu/ lucid multiverse
deb-src http://apt-cacher:9999/ubuntu/ lucid multiverse
deb http://apt-cacher:9999/ubuntu/ lucid-updates multiverse
deb-src http://apt-cacher:9999/ubuntu/ lucid-updates multiverse
## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu lucid partner
# deb-src http://archive.canonical.com/ubuntu lucid partner

deb http://apt-cacher:9999/ubuntu-security lucid-security main restricted
deb-src http://apt-cacher:9999/ubuntu-security lucid-security main restricted
deb http://apt-cacher:9999/ubuntu-security lucid-security universe
deb-src http://apt-cacher:9999/ubuntu-security lucid-security universe
deb http://apt-cacher:9999/ubuntu-security lucid-security multiverse
deb-src http://apt-cacher:9999/ubuntu-security lucid-security multiverse

After you have have saved your sources.list, run the following command:

sudo apt-get update

This will download the package index from the apt-cacher server. You should see lots of "Get" "Hit" and "Ign" lines coming from http://apt-cacher (and other lines for partner and 3rd party servers) if everything is working right. Remember that the update command will still be running at the normal speed because it has to fetch the indexes from the Internet every time to determine if the cache needs to download any new files. Also, the first time a given package is downloaded will still be at normal speed, as well.

To try it out, run sudo apt-get upgrade to download any available updated packages, or just install a new package of your choice. The apt-cacher should be functioning from the command line tools, from Update Manager, and from Synaptic Package Manager. Just remember not to adjust your repository checkboxes under the "Ubunto Software" tab in Synaptic's Settings:Repositories menu, because it no longer knows which repositories we've enabled (they are visible, however, on the "Other Software" tab, and you may adjust them there.)

How To Switch Locations

If you are using a laptop or netbook, you may appreciate the ability to quickly switch from one loication to another, and even to be able to download updates when you are not near your regular apt-cacher-ng server at all.

I have come up with a strategy to accomplish this. All you need to do (and you must do this while your current apt-cacher-ng server is accessible), is repeat the above steps for "Setting up the APT Cacher Server" on your local machine, then you can simply edit your /etc/hosts file and change the entry for apt-cacher to point at when you're on the run, or back to your server's IP when you are at home or work. Making that one line change will allow you to continue using all of the APT tools smoothly.

There is one disadvantage, especially for netbooks or devices with smaller hard drives, and that is that you will be storing two different caches on your hard drive for files which were downloaded while "on the go", one in the apt-cacher folder, and the other in the system's regular apt system cache under /var/cache/apt

Because of this, you may want to occasionally issue the command:

sudo apt-get clean

This will clean the system cache of installed packages out of /var/cache/apt, saving some disk space. This is generally a good idea to do on any Ubuntu system with disk space limitations, whether or not you are running apt-cacher-ng.

Also, if you ever need to delete the files under /var/cache/apt-cacher-ng, you may safely do so, and apt-cacher will download them again the next time they are needed.

Congratulations on successfully setting up apt-cacher-ng -- I hope! It has saved hours and hours of downloading here in our classroom.

Apt-Cacher-NG Client side perl script

First off a short explanation of Apt-Cacher-NG (acng) is in order. Based on a similar project simply called Apt-Cacher, acng is a download proxy for software packages, primarily for Debian/Ubuntu clients. (http://freshmeat.net/projects/acng/)
The way it works is the server at Community Computer Center, goes out to the official package and release servers and downloads the latest updates, that way when I, or anyone else wants to download updates, our computers simply have to download from the CCC server over the lan, which makes the download about ten times faster.

(Editorial Note: If you haven't set up apt-cacher-ng yet, follow this post first.)

The only downside is whenever I want to download the updates and I'm not at CCC, I have to go and edit the hosts file in /etc, so I wrote a perl script to cut down on the steps.

Copy the code into a text file and save it as a .pl, then to use, type sudo perl yourfile.pl.
While this is not the most elegant solution, it was my first attempt at using perl, enjoy.

#!usr/bin/perl -w
#script by @robots_unix

print "Are you at CCC?";
$name = ;
chomp ($name);
if ($name eq "yes"){ #When I'm at CCC
open(INFILE, '>/etc/hosts');
print INFILE " localhost uname
apt-cacher_ip apt-cacher
# apt-cacher
apt-cacher_ip server name
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts\n";
}else{ #When I'm not at CCC
open(INFILE, '>/etc/hosts');
print INFILE " localhost uname
#apt-cacher_ip apt-cacher apt-cacher
apt-cacher_ip servername
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts\n";

Saturday, July 10, 2010

Mutt and gmail

I think gmail is cool but it could be quicker to use. Right now it is slow and if you use the html version it lacks shortcuts. I think the cool way to go is to install mutt and use it to read your mail from the terminal. This article talks about some of the reasons why it is nicer and some methods to use to make mutt notify you about new mail. And how you might be able to use gmail contacts inside mutt. I think it would be cool to use googlecl to grab a contact while in mutt.
install mutt with gmail through IMAP
I like the fact you can make your own shortcuts in mutt that go to certain folders that you have set up like gmail has.
the following will make it so you go to the inbox when typeing gi and others.
bind editor noop
macro index gi "=INBOX" "Go to inbox"
macro index ga "=[Gmail]/All Mail" "Go to all mail"
macro index gs "=[Gmail]/Sent Mail" "Go to Sent Mail"
macro index gd "=[Gmail]/Drafts" "Go to drafts"

all the rest of the options are listed in the link above. I like the fact that you can change the editor for mutt too.
and the wiki might be helpful
if anyone tries to install gmail or another client with mutt then post here and let us know how it went.

Friday, July 9, 2010

Too many messages in mutt

I have learned something useful for anyone that uses mutt to mail from the terminal. I can be used to get messages from cron that runs periodically. Cron can run as much as once a minute. My cron has been sending me the message "don't update" for 6 months. It has caused about 40MB or 29000 messages to pile up in mutt. Now I have fixed the problem so it won't echo out for success. It should only tell me about failures not successes since they don't happen very often.

Usually you can delete hundreds of messages by holding down the delete button but since I had so many messages I had to learn how to delete bult amounts of mutt mail all at once. Here is how to do it:
type D
This will tell mutt to delete using a certain pattern. I tried using "update" but it didn't work so I figured out what will tag all of the messages at once.
type .
Thats right, a period.
Then type $ to have mutt ask if you want to purge 29000 messages. It takes a second and its done.