Last week, Susie and I provided the topic for the small Quaker meeting we've been attending as visitors (neither of us are officially Quakers). The topic we picked: Open Source.
The results? Spectacular. Almost everyone in the group understood and embraced the philosophy of Open Source. Many went home after the meeting to dabble with Open Source products, or followed up by asking questions. We also provided Ubuntu cookies as refreshments.
While there are some actual Open Source religious movements out there it is impressive to know that the Religious Society of Friends seems to take so well to the idea.
Saturday, December 30, 2006
Sunday, December 10, 2006
Pericles Lives!
Pericles is now open for business! It was only down for an hour, coming back online at 1:14. Everything I set out to complete was done, but I had to scramble to fix a few loose ends. Namely, I forgot to unpack one of the websites during the prior step because its archive had been placed in the wrong folder.
The system is running much faster. I'm sure there are still mistakes, so I'm going to go through and test each site meticulously to make sure things are smooth.
Hurray for Open Source!
The system is running much faster. I'm sure there are still mistakes, so I'm going to go through and test each site meticulously to make sure things are smooth.
Hurray for Open Source!
Saturday, December 9, 2006
Transferring the Hosted Content
Well, here I am. I took a break from Friday evening until this evening and then got back to work, and I have just finished the most terrible and nasty job of transferring the hosted site content over to Pericles.
Compress, copy, uncompress, change a few paths, add to /etc/hosts to "trick" the server into thinking the DNS is pointed here already when its still really on the old box, and edit /etc/apache2/pericles.conf (I include this from the main httpd.conf file) to tell it to pull from 192.168.0.1 instead of the real VirtualHosting IP. Run apache2 -k restart, and try it out in Firefox. Wash, Rinse, Repeat. About 35 times.
One of the things I had to do was set permissions on the web files so that my scripts were able to write to the folders they needed to. In Ubuntu, the default Apache2 user seems to be www-data.
After testing all the sites, it is now time to set up the necessary FTP accounts with access to their proper folders, to change the IP *back* to the real one in the pericles.conf, to install a fresh grab of the mysql database, to change the IP of the machine itself, and then to physically swap boxes and bring the new server online. After that, I expect chaos :-) Just kidding. I think I've checked everything out. I'm also limiting myself to finish this in the next 3 hours or else to hold off until another night so that this operation doesn't run into morning time.
I'll report back again once more progress has been made.
Compress, copy, uncompress, change a few paths, add to /etc/hosts to "trick" the server into thinking the DNS is pointed here already when its still really on the old box, and edit /etc/apache2/pericles.conf (I include this from the main httpd.conf file) to tell it to pull from 192.168.0.1 instead of the real VirtualHosting IP. Run apache2 -k restart, and try it out in Firefox. Wash, Rinse, Repeat. About 35 times.
One of the things I had to do was set permissions on the web files so that my scripts were able to write to the folders they needed to. In Ubuntu, the default Apache2 user seems to be www-data.
After testing all the sites, it is now time to set up the necessary FTP accounts with access to their proper folders, to change the IP *back* to the real one in the pericles.conf, to install a fresh grab of the mysql database, to change the IP of the machine itself, and then to physically swap boxes and bring the new server online. After that, I expect chaos :-) Just kidding. I think I've checked everything out. I'm also limiting myself to finish this in the next 3 hours or else to hold off until another night so that this operation doesn't run into morning time.
I'll report back again once more progress has been made.
Ubuntu Cookies
I saw this post and decided that the world needs more Ubuntu cookies.
I actually wrote a very long and detailed post about how I made these, *but* through some freak accident, I selected all my text, opened the context menu, and selected 'insert dummy lorem ipsum'. Firefox+Blogger wouldn't let me undo, so, you get the abridged version now.
I started out by mixing up the dough using the recipe provided by Joseph Hall. It turned out the consistency of frosting... I've made sugar cookies before, and the dough was more like soft play-dough, I added 3/4 cup more flour but it didn't help. Before I chilled the dough, I added the food coloring that way the dye could be dispersed evenly.
I'm doing the whole thing with a minimum of special tools, just my hands, parchment paper, four and a large cutting board (and something round I still have to find ).
Since the dough was so soft, it was a lot easier to roll the logs, then place them on the parchment paper, then flatten them. Otherwise they kept breaking apart as I moved them.
The main thing I didn't care for in Joseph Hall's version of these cookies was the mid-section, it was too big in proportion to the rest of the pieces compared to the actual logo. So, I made sure to make mine smaller, although I hope it's not too small.
I started at 8:00pm, it's now 10:20pm - time to roll out the piece-between-the-head-and-arms. After they freeze, it'll be time to assemble the whole thing.
I accidentally tried to add the impressions for the head into the main section .. oops.
10:40pm, If you ever do anything like this, make sure to measure the circumference of your center log and figure out how wide to make the wrapper. (hmm, I should probably do that for the outer wrapper) I got it right on the third try. Luckily the dough was stiff enough for me to unroll it when I got it wrong. I have a little yellow left, but the other pieces were all the right length.
11:15pm, I took the pieces for between the head and arms out and let them thaw while I added the indentations to the main log. I started by using a dry erase marker, but ended up using my thumbs. The dividing piece was hard to stick on because of all the flour I had used - but I don't think I could have rolled out the dough without the flour, so perhaps some water would have helped stick the pieces all together. I just hope the final product doesn't fall apart when I cut it.
Adding the heads was easy, even though they kept snapping. I kept the whole thing rolled up in parchment paper to hold it together while it freezes up again.
I can see that the final wrapping piece will need to be wider, so I'm thawing it out while I type this.
11:30pm, all finished! Now I understand the large center piece! it would have been much easier, but I think I'd still rather have the logo be the right proportions. I'll let it freeze for an hour, then cook a few and see how they worked out.
12:30pm, the dough was pretty solid, so I cut off four inches from the "bad end" where all the ends weren't the same size, sliced them and stuck them in the oven. The biggest problem was that the various sections didn't want to stick together. Part of the reason they won't stick together is because they were all frozen pieces, and there wasn't any soft dough to stick too. My solution is to set a piece of the log out so it can all thaw, then solidify it again.
12:45pm, BLAST! the cookies spread! (a lot) I think there was either something very wrong with the recipe, or there's something very wrong with one of my ingredients.
I'll have to try this again with a recipe of dough I know doesn't spread.
I actually wrote a very long and detailed post about how I made these, *but* through some freak accident, I selected all my text, opened the context menu, and selected 'insert dummy lorem ipsum'. Firefox+Blogger wouldn't let me undo, so, you get the abridged version now.
I started out by mixing up the dough using the recipe provided by Joseph Hall. It turned out the consistency of frosting... I've made sugar cookies before, and the dough was more like soft play-dough, I added 3/4 cup more flour but it didn't help. Before I chilled the dough, I added the food coloring that way the dye could be dispersed evenly.
I'm doing the whole thing with a minimum of special tools, just my hands, parchment paper, four and a large cutting board (and something round I still have to find ).
Since the dough was so soft, it was a lot easier to roll the logs, then place them on the parchment paper, then flatten them. Otherwise they kept breaking apart as I moved them.
The main thing I didn't care for in Joseph Hall's version of these cookies was the mid-section, it was too big in proportion to the rest of the pieces compared to the actual logo. So, I made sure to make mine smaller, although I hope it's not too small.
I started at 8:00pm, it's now 10:20pm - time to roll out the piece-between-the-head-and-arms. After they freeze, it'll be time to assemble the whole thing.
I accidentally tried to add the impressions for the head into the main section .. oops.
10:40pm, If you ever do anything like this, make sure to measure the circumference of your center log and figure out how wide to make the wrapper. (hmm, I should probably do that for the outer wrapper) I got it right on the third try. Luckily the dough was stiff enough for me to unroll it when I got it wrong. I have a little yellow left, but the other pieces were all the right length.
11:15pm, I took the pieces for between the head and arms out and let them thaw while I added the indentations to the main log. I started by using a dry erase marker, but ended up using my thumbs. The dividing piece was hard to stick on because of all the flour I had used - but I don't think I could have rolled out the dough without the flour, so perhaps some water would have helped stick the pieces all together. I just hope the final product doesn't fall apart when I cut it.
Adding the heads was easy, even though they kept snapping. I kept the whole thing rolled up in parchment paper to hold it together while it freezes up again.
I can see that the final wrapping piece will need to be wider, so I'm thawing it out while I type this.
11:30pm, all finished! Now I understand the large center piece! it would have been much easier, but I think I'd still rather have the logo be the right proportions. I'll let it freeze for an hour, then cook a few and see how they worked out.
12:30pm, the dough was pretty solid, so I cut off four inches from the "bad end" where all the ends weren't the same size, sliced them and stuck them in the oven. The biggest problem was that the various sections didn't want to stick together. Part of the reason they won't stick together is because they were all frozen pieces, and there wasn't any soft dough to stick too. My solution is to set a piece of the log out so it can all thaw, then solidify it again.
12:45pm, BLAST! the cookies spread! (a lot) I think there was either something very wrong with the recipe, or there's something very wrong with one of my ingredients.
I'll have to try this again with a recipe of dough I know doesn't spread.
Wednesday, December 6, 2006
Migrating MySQL Data
Its time to migrate a copy of the MySQL data so that I can begin testing the websites running on Pericles in a closed environment before I let this out into the wild.
The data lives in /var/lib/mysql
I'm cool with that, but I'm going to replace it with my existing data, so first I'll rename it to preserve the original:
Next, I unpack my MySQL backup, which was archived from my replication slave a couple days ago. Since I'm going to make this into the new master, I'll first use this data as a testbed and afterwards I'll replace it with a fresh copy when I'm really ready to switch masters. My data is currently about 1 gigabyte, which shrinks to 111 megabytes when compressed, so it takes about five minutes to copy and longer to unpack. I should point out that innobase and myisam data on Linux and Windows are "binary compatible" so they can be copied directly across the board, but you need to make sure your mysql configuration files have the same innodb settings in them, or else. Oh yeah, I should also mention that the MySQL version numbers are both very close, and up to date, so the fields in the user tables and so forth are the same format. I wouldn't recommend migrating a MySQL 4.0 database to a MySQL 5.0 environment using this technique because the user fields changed a bit.
I deleted the master.info file so that this server won't try to connect to the replication master like the replication slave from which it was copied was programmed to do. I'll configure it to be a replication master once I do the real thing.
I had to change my bind-address=0.0.0.0 in /etc/mysql/my.cnf in order to get the MySQL to bind to all network interfaces (I need to be able to connect from several networks.) This is ok, because the mysql users table has restrictions on which users are allowed to connect from which hosts. I wanted to change it in /var/lib/mysql/my.cnf which seemed to be recommended by the heading in /etc/mysql/my.cnf, but copying the cnf file and changing it there did not make any difference, so I resorted to changing the /etc/mysql/my.cnf version of the file.
Starting up MySQL I see this:
Now I need to add back in the debian-sys-maint user that I lost by overwriting the user tables. This is easy, because they leave the random password sitting unencrypted in /etc/mysql/debian.cnf
I did this by logging into MySQL from the command line interface as root, and executing the following commands:
I now stopped and started the MySQL server:
This worked like a charm; I no longer see the error message, which means debian-sys-maint can do its magic.
sudo /etc/init.d/mysql stop
The data lives in /var/lib/mysql
I'm cool with that, but I'm going to replace it with my existing data, so first I'll rename it to preserve the original:
cd /var/lib
sudo mv mysql mysql-original
sudo mkdir mysql
sudo chown mysql.mysql mysql
Next, I unpack my MySQL backup, which was archived from my replication slave a couple days ago. Since I'm going to make this into the new master, I'll first use this data as a testbed and afterwards I'll replace it with a fresh copy when I'm really ready to switch masters. My data is currently about 1 gigabyte, which shrinks to 111 megabytes when compressed, so it takes about five minutes to copy and longer to unpack. I should point out that innobase and myisam data on Linux and Windows are "binary compatible" so they can be copied directly across the board, but you need to make sure your mysql configuration files have the same innodb settings in them, or else. Oh yeah, I should also mention that the MySQL version numbers are both very close, and up to date, so the fields in the user tables and so forth are the same format. I wouldn't recommend migrating a MySQL 4.0 database to a MySQL 5.0 environment using this technique because the user fields changed a bit.
I deleted the master.info file so that this server won't try to connect to the replication master like the replication slave from which it was copied was programmed to do. I'll configure it to be a replication master once I do the real thing.
I had to change my bind-address=0.0.0.0 in /etc/mysql/my.cnf in order to get the MySQL to bind to all network interfaces (I need to be able to connect from several networks.) This is ok, because the mysql users table has restrictions on which users are allowed to connect from which hosts. I wanted to change it in /var/lib/mysql/my.cnf which seemed to be recommended by the heading in /etc/mysql/my.cnf, but copying the cnf file and changing it there did not make any difference, so I resorted to changing the /etc/mysql/my.cnf version of the file.
Starting up MySQL I see this:
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)'
Now I need to add back in the debian-sys-maint user that I lost by overwriting the user tables. This is easy, because they leave the random password sitting unencrypted in /etc/mysql/debian.cnf
I did this by logging into MySQL from the command line interface as root, and executing the following commands:
USE mysql;
CREATE USER 'debian-sys-maint'@'localhost';
UPDATE user SET password = PASSWORD('PasswordCopiedEarlier') WHERE user = 'debian-sys-maint';
GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost';
FLUSH PRIVILEGES;
I now stopped and started the MySQL server:
sudo /etc/init.d/mysql stop
sudo /etc/init.d/mysql start
This worked like a charm; I no longer see the error message, which means debian-sys-maint can do its magic.
Converting Hardcoded Pathnames
The next step on my adventure with Pericles is to convert all of the hard-coded pathnames from F:\Web to /var/www (as well as any other hardcoded paths I find.) I've been pretty careful not to use hard-coded names in unnecessary places, putting most of them in simple configuration include files and so forth, because I knew from the beginning that this day would come.
I typed grep and it turns out that I have a convenient version of Turbo grep (comes with any Insprise/Borland product of the last decade), but GNU grep should work just as well. The syntax for turbo grep that I'm using is:
grep -di "f:" *.ph*>hardcoded
I'm using *.ph* because I am searching all *.php and *.ph files. I typically use *.ph for include files instead of *.inc, the -di means search subdirectories and ignore case, and I keep my data and web files on the F: drive. I'm redirecting the output to a file named hardcoded so that I can use it as a checklist. I'll do the same thing again but, like this:
grep -di "c:" *.ph*>>hardcoded
To get all references to anything on the C: drive (I know I refer to an executable font conversion tool from one place, for example.) and the double angle brackets mean to append the result to my previous file instead of overwriting.
Now that I have my checklist, here's the plan:
I want to convert all the code while still on the Windows box to support Linux based on a single configuration change. I will make an include file with one function in it that takes any path given as a parameter and if it starts with /var/www, to convert it back to F:/Web. This will allow Linux to be the native language with a "patch" for Windows support. This should have been part of my plan from day 1 (two years ago), but it didn't occur to me. The idea is that I can replace this function with an "identity" function such as: function linpath($a) { return $a; } in order to use raw Linux paths instead of Windows.
I will also do similar adaptations on a situational basis as I run into any F: or C: references that can't be solved with the above include file.
I anticipate some difficulties with file permissions since Apache on Linux is more secure than Apache on windows. My main resolution to this will be to consolidate the affected items that need to be accessed into consistent areas that the Apache process has access to read. If I have any chroot issues, I can overcome them with the use of mount --bind as explained in my article on setting up FTP.
This path translation will probably be the most tedious task in the Pericles project. I'll report back when I have finished.
4:18 P.M. - Upon more deliberate consideration, I have determined to abandon this undertaking and instead to change the paths manually. Why? There only seems to be about one per website, with an odd exception here or there. The trouble crops up when deciding exactly how to include the file with the path mapping function without using a path for itself. Furthermore, even if I install it into the default includes path, it seems silly to include a file only to help correctly include another file. May as well just change them when the time comes.
I typed grep and it turns out that I have a convenient version of Turbo grep (comes with any Insprise/Borland product of the last decade), but GNU grep should work just as well. The syntax for turbo grep that I'm using is:
grep -di "f:" *.ph*>hardcoded
I'm using *.ph* because I am searching all *.php and *.ph files. I typically use *.ph for include files instead of *.inc, the -di means search subdirectories and ignore case, and I keep my data and web files on the F: drive. I'm redirecting the output to a file named hardcoded so that I can use it as a checklist. I'll do the same thing again but, like this:
grep -di "c:" *.ph*>>hardcoded
To get all references to anything on the C: drive (I know I refer to an executable font conversion tool from one place, for example.) and the double angle brackets mean to append the result to my previous file instead of overwriting.
Now that I have my checklist, here's the plan:
I want to convert all the code while still on the Windows box to support Linux based on a single configuration change. I will make an include file with one function in it that takes any path given as a parameter and if it starts with /var/www, to convert it back to F:/Web. This will allow Linux to be the native language with a "patch" for Windows support. This should have been part of my plan from day 1 (two years ago), but it didn't occur to me. The idea is that I can replace this function with an "identity" function such as: function linpath($a) { return $a; } in order to use raw Linux paths instead of Windows.
I will also do similar adaptations on a situational basis as I run into any F: or C: references that can't be solved with the above include file.
I anticipate some difficulties with file permissions since Apache on Linux is more secure than Apache on windows. My main resolution to this will be to consolidate the affected items that need to be accessed into consistent areas that the Apache process has access to read. If I have any chroot issues, I can overcome them with the use of mount --bind as explained in my article on setting up FTP.
This path translation will probably be the most tedious task in the Pericles project. I'll report back when I have finished.
4:18 P.M. - Upon more deliberate consideration, I have determined to abandon this undertaking and instead to change the paths manually. Why? There only seems to be about one per website, with an odd exception here or there. The trouble crops up when deciding exactly how to include the file with the path mapping function without using a path for itself. Furthermore, even if I install it into the default includes path, it seems silly to include a file only to help correctly include another file. May as well just change them when the time comes.
Fixing rndc error with bind9
While setting up DNS on Pericles I did run into one snag:
rndc: connect failed: connection refused
The first thing to know, is that in Ubuntu Server's default setup, the /var/log/syslog file contains errors relating to bind and rndc startup. I watched this file and found some syntax errors as I tried to resolve this problem, and I recomend you do the same. Here is my solution:
Careful, this will overwrite your rndc.conf (run this while in the /etc/bind folder):
Open the new rndc.conf. First, take the hyphen out of all the rndc-key names. I don't know why. rndckey is what I ended up needing. I think maybe bind9 removed support for the hyphen in these names, but rndc-confgen didn't know it.
Now, copy the bottom section (the commented out part) out, we're going to paste it into the top of named.conf.local, and uncomment it. Save changes to rndc.conf, and after pasting the other section into named.conf.local change the part inside the allow { } to localhost; instead of 127.0.0.1, for similar reason as before it seems to want a name instead of a hardcoded IP in this version.
I killed the named process then tried starting it again, and once again. I found an error in my syntax by looking at /var/log/syslog, and after fixing this it worked without any error.
rndc: connect failed: connection refused
The first thing to know, is that in Ubuntu Server's default setup, the /var/log/syslog file contains errors relating to bind and rndc startup. I watched this file and found some syntax errors as I tried to resolve this problem, and I recomend you do the same. Here is my solution:
Careful, this will overwrite your rndc.conf (run this while in the /etc/bind folder):
rndc-confgen > rndc.conf
Open the new rndc.conf. First, take the hyphen out of all the rndc-key names. I don't know why. rndckey is what I ended up needing. I think maybe bind9 removed support for the hyphen in these names, but rndc-confgen didn't know it.
Now, copy the bottom section (the commented out part) out, we're going to paste it into the top of named.conf.local, and uncomment it. Save changes to rndc.conf, and after pasting the other section into named.conf.local change the part inside the allow { } to localhost; instead of 127.0.0.1, for similar reason as before it seems to want a name instead of a hardcoded IP in this version.
I killed the named process then tried starting it again, and once again. I found an error in my syntax by looking at /var/log/syslog, and after fixing this it worked without any error.
Subscribe to:
Posts (Atom)