...making Linux just a little more fun!
Tux has been Linux's mascot since shortly after the release of the
2.0 kernel. He's been known to be quite the fellow for the ladypenguins,
and
A Brief History Of Tux - So
Far will surely tell you more than you needed to know about this
dashing fellow. All too often, it's a working vacation for him, but he's
considering a hiking
trip or a cruise sometime this
Summer.
Heather is Linux Gazette's Technical Editor and The Answer Gang's Editor Gal.
Heather is a hardware agnostic, but has spent more hours as a tech in
Windows related tech support than most people have spent with their computers.
(Got the pin, got the Jacket, got about a zillion T-shirts.) When she
discovered Linux in 1993, it wasn't long before the home systems ran Linux
regardless of what was in use at work.
By 1995 she was training others in using Linux - and in charge of all the
"strange systems" at a (then) 90 million dollar company. Moving onwards, it's
safe to say, Linux has been an excellent companion and breadwinner... She
took over the HTML editing for "The Answer Guy" in issue 28, and has been
slowly improving the preprocessing scripts she uses ever since.
Here's an autobiographical filksong she wrote called
The Programmer's Daughter.
Heather got started in computing before she quite got started learning
English. By 8 she was a happy programmer, by 15 the system administrator
for the home... Dad had finally broken down and gotten one of those personal
computers, only to find it needed regular care and feeding like any other
pet. Except it wasn't a Pet: it was one of those brands we find most
everywhere today...
What Fools We Linuxers BeWe're thinking of a very special edition of "Foolish Things We Do With Our Computers". If you did something foolish, or had something foolish inflicted upon you by your otherwise favored silicon lifeform, and you're willing to have just a little more fun with it in public, send it along to us.
This does not have to be in article format. You can just send it to
The Answer Gang (tag, at our happy domain linuxgazette.net) with the
subject "Foolish Things With Computers" and we'll gather them all up and
roast marshmallows over them. Let us know if you want your name left
in - we'll take these anonymously too. But don't just make them up; we
want real tales that make us groan and go agh I won't do that again!
Articles about voiceIn the issues numbered about 85 to 89 we had articles about the use of voice on computers in developing areas, for hadicapped usage, and the like.
Sound systems on Linux have improved considerably - we've had Jimmy cover "Songs In the Key of Tux" - but we haven't come back to the topic of ordinary voices. Voice Over IP got a mere nod in my Answer Gang blurb last Summer.
If you are interested in writing an article on this topic, see our article submission guidelines and then drop us a line!
Knight's tourSent to our author Kapil Hari Paranjape privately, printed with permission. -- Heather
Hello,
and thanks for your amusing article about the Knight's tour in Linux Gazette. I don't understand Python, but I have some experience of the Knight's tour problem. Mathematicians might have annoying habits, and one could be to end their articles with Exercises Like: 4.There is apparently a better algorithm than Warnsdorff's for the Hamiltonian circuit. Find it and implement it.
"apparently" ?! Enough to drive at least me crazy
with some decently efficient Java applets and some about the research done about the problem of how many closed tours there are. (At least I would call that problem 'hard')
Once again, Thanks for your article
and Best regards
Gunno Törnberg
RW "Quick Format"...Here's a reply to my posting of the Packet Writing material on my own site. I've gotten his permission to forward this into TAG as commentary on the original, so if you want to use it... ?
best,
.brian
Hi Brian,
Actually the time for UDF formatting of a CDRW is consistant with Nero's packet driver (InCD) under Windows. InCD used to have multiple selections of "Format" and "Quick Format" available but only the "Format" option was available for a BLANK CDRW disk and formatting could take up to 45min to an hour. A previous formatted UDF disk would let you "quick format" and take considerably less time; some times as short as 6min.
Of course, I suspect the "quick format" is really only doing a quick erase and random verify of the file system. This would be similar to a 'quick erase' of a CDRW disk which was written/formatted as an iso -- erase the header and directory structure but don't bother with rest of data (i.e. lets hope the surface, etc. is OK and we'll just overwrite for the new compilation).
I had a Mount Ranier capable CDRW drive at one point and I noticed that it worked a bit differently. Apparently MR drives can format and write "at the same time" and also do formatting in the background. So, when using MR (instead of UDF 1.5) it appeared only the disk headers and directory structure was initially formatted. Then, as data was sent to the drive (by "drag and drop" or whatever) it was cached and buffered, then the space needed was formatted and written to in the background. When the disk was ejected a significate delay occured while anything left in the cache was written out to disk and the disk cleaned up. It appeared that only as much of the disk was actually formatted as needed because you could "force" a full disk format in MR and it would take about as long to complete as a format as UDF.
Users should also be aware that UDF file systems are much, much less safe than standard iso compilations. They are more effected by heat. Not all UDF file systems are equal -- especially now that various revision levels are out (UDF1.5 appears to be "standard recommended" while there were revisions up to 2.5 last time I looked). In practice I've found that using packet writting is tends to work only for the computer/drive you write it on and only for relatively short term storage.
Note the fact that Windows XP's setup for CD writing uses a disk buffer and writes only a iso. It simulates a "drag and drop" random access file system by mapping the CDRW disk to the buffer (actually a system folder called "cd burning") and then just burns asks to burn a standard iso. For adding to a written disk it appear to load what is already on disk to the buffer adding to the new files then erase the CDRW disk and re-burn it. I suspect even good old Microsoft figured the odds of including UDF packed writing would be adding another can of worms to XP.
David Yerka
PS: I have been testing Xandros OC 3.0.1 and I find I like it quite a bit. I didn't find 2/2.5 really ready for an average business user but Xandros 3 looks to be a true MS desktop killer. A number of my clients are fed up with paying through the nose for Windows "upgrades" -- some really feel MS tugging the chain -- so they are very interested. I even have one office where the practice management application will run under Crossover Office -- and the developers have decided to commit to a Linux version for release next year.
Rob Tougher's article, issue 96Dear editor,
This was pretty good article but it seems to leave out something. Because of that I've had hours of pain. Perhaps a note could be added.
Here is what seems to be missing: setting up AXIS_HOME, AXIS_LIB, and AXIS_CLASSPATH, as in
set AXIS_HOME=c:\axis set AXIS_LIB=%AXIS_HOME%\lib set AXISCLASSPATH=%AXIS_LIB%\axis.jar;%AXIS_LIB%\commons-discovery.jar; %AXIS_LIB%\commons-logging.jar;%AXIS_LIB%\jaxrpc.jar;%AXIS_LIB%\saaj.jar; %AXIS_LIB%\log4j-1.2.8.jar;%AXIS_LIB%\xml-apis.jar;%AXIS_LIB%\xercesImpl.jar ;%AXIS_LIB%\wsdl4j.jar
Regards and keep on going,
Bill Bruns
Trojan files on TLDP server?Forwarded from a discussion on the Linux Documentation Project mailing lists. Linux Gazette is a member of TLDP... -- Heather
Date: Mon, 4 Apr 2005 15:02:12 +0000 (UTC) From: Machtelt Garrels <address_elided>
Can somebody look into this? It never happened to me... Please confirm if this is fake or not.
Tille.
|
............... Date: Mon, 28 Mar 2005 16:57:59 -0800 From: Brian Wildasinn <address_elided> Hello TLDP, URGENT NOTICE: Trojan LG TLDP archives alert! On March 25, 2005, I download some files from TLDP. My download script shows a time stamp of 9:10PM, which downloaded the entire ftpfiles directory at linuxgazette from my home LAN at 66.218.50.80. I have a WinXP notebook attached to a wireless access point. After downloading some Linux Gazette tarballs from http://linuxgazette.net/ftpfiles, my security scanners show an active suspicious port open. I could telnet into port 5400 on WinXP from my FreeBSD box over my LAN. Using NMAP security scan on FreeBSD `nmap -sS -P0 <wireless access point w/WEP encryption/router MN-700>` showed port 5400 open, which is described as "5400/tcp excerpt Excerpt Search" or Bladerunner Trogan. Here is the results of ClamWin from my infect notebook: See attached ClamWin.infection-log.txt ............... |
Date: Mon, 4 Apr 2005 10:27:01 -0700 From: Rick Moen <address_elided>
Quoting Machtelt Garrels:
Can somebody look into this? It never happened to me...
Please confirm if this is fake or not.
What you have there is a pair of false positives. I'm not sure what in Linux Gazette issue 86 Clamwin thought was an instance of "Exploit.IFrame.Gen", which I gather is an MS-Outlook exploit. The Gazette has a feature near the end of many issues where the text of particularly hilarious spam and/or virus mail is published and mocked, so that might well be it.
I would guess that the "HTML.Phishing.Bank-1" Clamwin thought it found, I'd guess it was (likewise) erroneously triggering on the "Spam Cuteness" item in Jimmy O'Regan's "Linux Launderette" column.
Brian, there's nothing wrong with paying close attention to your anti-virus software if you're on MS-Windows, but you'll want to read the results with at least a little skepticism: For one thing, given that the Linux Gazette files are a magazine, and that you read the contents rather than executing it as a program, it's unclear to me how -- even if every issue were packed chock-a-block with MS-Windows worms, trojans, viruses, and exploits -- those could have been anything but inert curios.
You may indeed have an alarmingly open "port 5400" on your MS-Windows XP box, of course -- for entirely independent reasons. Good luck with that.
|
...making Linux just a little more fun! |
The Answer Gang
![]() By Jim Dennis, Jason Creighton, Chris G, Karl-Heinz, and... (meet the Gang) ... the Editors of Linux Gazette... and You! |
We have guidelines for asking and answering questions. Linux questions only, please.
We make no guarantees about answers, but you can be anonymous on request.
See also: The Answer Gang's
Knowledge Base
and the LG
Search Engine
Greetings from Heather SternHello folks, things are a bit hectic over here at ye olde Answer Gang. It's been a complete madhouse - no wait, an incomplete madhouse -
Just check out the Lifehacks we have this month. Hopefully they'll keep your life less tangled than ours. We're saving up our Tips and our threads from The Answer Gang about Linux itself will be back next month, after we get the backhoes rolled away, and the forklifts are gone... Sorry for the dust, folks.
There's a friend of mine who says, when tech does poorly, buy stocks in raw materials for construction. After all we know the backhoe is the real enemy of the internet! I wonder if it's time to sell those puppies - it looks like things in the tech world are pretty much looking up. Not necessarily all at once, but certainly, there are signs of Spring. Renewal is here, there are random() seeds everywhere. We're getting to the /root of things (with a bit of hack and /-ing, true). Duck season, wabbit season... allergy season.
Ok, enough of that, I'm hiding indoors and doing programming until
Summer.
Share drinks with your friendly local bartender in the TAG lounge, and
this is what you get:
Happiness is like water. It might come back as rain, but it's always there if you look for it.
Diving into something new, or coming out of deep waters? Sploosh.
Remember to protect your silicate buddies from the elements.
See you next month, folks
p.s. Yes, we do celebrate April Fools Day around here. Please send us any Foolish Things You Did With Your Computer for a special run in next month's issue. Any particularly yummy Answers you have discovered, are also welcome; by this means you, too, can join The Answer Gang! Send your submissions to: tag .at. linuxgazette.net - Thanks!
p.p.s. Yes, questions are good too, we're always eager to answer you. Well, almost always. We don't guarantee an answer (tho stumpers are likely to land in Help Wanted). However - please don't ask us things you don't want published - the Gang's not a private place. But you can be anonymous, you only need to ask that too. -- Heather
LifehacksFrom Jimmy O'Regan
Answered By: Jimmy O'Regan, Brian Bilbrey, Raj Shekhar, Heather Stern
I'm looking for everyone's lifehacks. For more information about what a lifehack is, scroll down -- I've included the introduction to an article I was writing for another site (cough) that I couldn't make relevant there -- but in short, it's things that make your life easier, like Ben's quote catcher or my screen scrapers (they save me ~5 minutes per day that I can waste on IRC instead.
So drag out your ugly scripts, or at least say what they do in principle. (No, I don't have any that haven't been made into articles already :) -- Jimmy
Lifehacks are an idea that was put forward by Danny O'Brien at the O'Reilly Emerging Technology Conference . Lifehacks are the result of taking one of the basic principles of self-help -- find out how the experts work, and emulate them -- and applying it to geeks.
One of the least surprising discoveries of the Lifehacks research is that "Overprolific Alpha Geeks" spend most of their working lives using the shell. Two of the conclusions of the talk are that geeks will have private blogs/RSS feeds.
Why would you use private blogs?
Most people need to provide their bosses with periodic reports. Because feed aggregators are available for almost every imaginable platform, reading feeds is not an issue. Having reports available in a private blog offers two immediate benefits: each report is archived in a known location; and the feed reader provides notification of new entries (as well as keeping its own archive of entries). Making reports available as a feed can also boost productivity and increase communication -- sysadmins, for example, can generate many of their reports from log files, and add this to the feed so their boss no longer needs to wait around for a formal report; programmers can use scripts in CVS (or other version control systems) that generate a report containing a list of changed files and the log message for every commit.
A popular use of blogs is as a whiteboard -- people thinking out loud, inviting passers-by to offer suggestions or criticism which may help to improve an idea. By maintaining different blogs with varying levels of availability, people can share ideas with the appropriate audience without revealing things they would prefer to keep secret.
The most basic advantage of writing down an idea is that you no longer have to worry about remembering it. Even writing to a completely private blog, as a "todo" list, has advantages. Writing an idea forces you to clarify your own thoughts: the simple act of writing an idea down to remember it can lead to you expressing it well enough to share it with a wider audience. Once an idea is written, it can then be refactored -- you can share it at a later date, when you have developed it further. Another advantage is that each blog entry is automatically dated, so each idea can be compared with public blog entries from the same time, so you can discover why the idea was relevant; and if nothing else, a blog provides you with a single place where you can archive your ideas and "secret scripts" in case you need them again.
Semi-public blogs, available only on a company or department intranet, allow co-workers to keep track of each others activities and to offer and receive suggestions about ideas without publicly exposing company secrets. Information put on a semi-private blog may later become suitable for public consumption, such as a note about a bug in a product, or information about server downtime.
[Brian] Thank you.
Thank you, Thank You, THANK YOU!
This is the sort of thing that I've been doing unconsciously for the last 20 years, and what's been lacking is the vision to make it a conscious process.
Every machine I have does webserving, I drop stuff into the serving space for testing, for review, for remembering.
I put stuff on my public site so that when I forget I ever knew how to do a thing, I can google for the answer and find what I wrote once upon a time (that's happened THREE times in the last year).
I rsync everything, everywhere. Cron takes care of most of it, occasionally I'll manually run a set to a sometimes running box, and then shut it down and put it in the basement. Mostly that's private stuff, and not the things that can be replicated or are static, committed to DVD+R (like the ogg collection).
I run phpGroupWare at work with Calendar, TroubleTicket and Todo modules active.
I publish fun and useful links in my webspace on the internal server - everything from a demo of how the Windows messenger error looks when Samba + ClamAV +Samba-vscan catches an inbound virus, to the mods I might make to my cube to get a nerf-weapon war type of battle-cube.
All of my email accounts are IMAP, I have 5 active places, and I copy stuff from account to account, in and out of the assorted todo sub-folders as I want to apply the topic to the part of my life in question.
[Jimmy] I do the same. Anything that I really like, or may need in real life, I email to my phone (my phone has Blackberry email, so email to my private account comes in like an SMS message: straight to the phone, no need to check :)
Email is one of the top geek ways of keeping information, according to that survey. Wanna remember something, mail it to yourself.
[Brian] I don't have a useful PDA. I've had FUN ones, linux-based, but I'm thinking I need a real one, one of these days. Useful AND used. All the Linux ones provided tools to sync/work with Windows boxen. Um, sorry, don't run that one, got any other choices? I'm leaning towards Treo. Any thoughts?
[Jimmy] I use my phone as a pda. Getting SyncML going is something I really want to do, but I'm not sure I want to be synchronising across the Internet when the phone is right beside the blasted PC :(
SyncML is now part of the Open Mobile Alliance (http://www.openmobilealliance.org/index.html) -- Heather
[Brian] I have one "blog" that I'm experimenting with, using Wordpress . There are parts of it I like, but for many purposes I really like the read-in-time-ordinal of journalling rather than the reverse mode of Blogging. I suppose that's why I've never really mentally connected with RSS feeds, even though a great number of people I know think that feeds are the greatest thing since sliced bread. Are these "problems" for me? Am I just in the wrong phase of a Candlestick-Two Faces conundrum? Thoughts, please...
I LOVE the idea of using a blog as a log watch aggregator. That ROCKS!
[Jimmy] Heh. That was an obvious use I didn't see mentioned. Back in college I used to have a bunch of scripts that ran grep over various log files, diffed them against yesterday's, and mailed the result to an admin account.
I used them to track things like who used su and when etc., and had something similar in root's .bash_logout so I could compare and see who was doing what (not to snoop, just to offer a helping hand to the newbie admins).
[Brian] What's in my ~/bin: 87 scripts, mostly ugly hacks to run CLI and GUI tools in a specific order. For example:
bilbrey@vimes:~/bin$ cat getpix | egrep -v "^$"
See attached brian.getpix.bash.txt
Since the USB card reader doesn't ALWAYS see media inserted, a quick cfdisk brings that problem to an end. I mount the card, copy the contents to the directory I specified on the commandline. Ick, no test for correct input, eh? Unmount, prompt, call another script to convert all to 640x480. use GQview to pick and choose. Those I keep, I may edit using GIMP from within GQview. When I'm done, I have the 640x480 snaps that I'll use on the website. Exit GQview and the script continues. Make thumbnails of the remaining files. Copy everything into my local copy of the webtree. Blow the whole processing tree away, leaving me with camera originals on the disk. I don't use this one much anymore, because I have a new camera and a new method of doing some of that stuff. I haven't automated it yet because I've been too busy to save time. Sigh.
[Jimmy] I keep my photos on flickr.com (and on CD, but they're easier to find on flickr), and my links on del.icio.us . In issue #110 I described what I use to get a montage of my last N photos ( Flickr and Perl ) and My tip in #112 has a script I use to get a Netscape bookmarks-compatible version of all the links I have in del.icio.us.
[Brian] I have another little script, a three liner that drops me down into the current year's working directory in my web space, pops a firefox window pointing at http://localhost/xxx/xxx/blah.php, and fires up bluefish with the most recently edited page in the window.
I have a number of scripts dedicated to setting up ssh tunnelling for VNC connections to windows desktops at my last-but-two employer out in CA, where I still consult from time to time, and it's much easier to answer Netscrape or LookOut questions when they can SHOW me instead of trying to describe things over the phone.
[Heather] For being able to show someone _else_ that VNC session, VNC snapshot over at sourceforge looks nice...
[Brian] I have lots of little scripts for rsyncing chunks of my local machine to other places, for web publishing, for backup, for sharing data.
I need to spend more time being efficient, writing still more tools to do my work for me.
[Jimmy] Same here. I'm finding more and more that it pays to take a step outside myself and really pay attention to what I'm doing, then use this function:
histdump() { history $(($1 + 1))|head -n $1|awk '{print $2}'>sh-$(date +"%b%d-%H:%M"); }
[Raj] One thing I use heavily is Emacs. It has a number of nifty modes which make life much organized for a programmer. I will list a few especially good ones
- emacs-PlannerMode:
- http://sacha.free.net.ph/notebook/wiki/PlannerMode.php#sec1
PlannerMode is an organizer and day planner for Emacs. It helps you keep track of your pending and completed tasks, daily schedule, dates to remember, notes and inspirations.
emacs-outline-mode: very useful if you are writing an outline for an article or just taking notes. (I find wiki-mode more user friendly, see below)
[Jimmy] I'm familiar with it :) (http://linuxgazette.net/108/oregan.html)
[Raj] emacs-wiki-mode: This is a Emacs mode for maintaining a local Wiki. Though I use it for making outlines of articles and keeping notes during meetings. The main idea is that you write using plain text, but it is rendered on emacs as html. (emacs-wiki-mode comes with emacs-planner-mode)
Instead of having a personal blog to keep a track of personal-todos/inspirations etc. , I use a wiki: http://rajshekhar.net/wiki
[Jimmy] Ah. There's something worth mentioning. You all know and love Heather's SysadMoin (http://www.starshine.org/SysadMoin), right?
Yeah, wikis are brilliant. You can even get them for your desktop now (http://www.beatniksoftware.com/tomboy)
[Heather] The great thing about MoinMoin is, it supports access control lists, so it can be public and private spaces at the same time.
Yeah, pardon my dust over there at SysadMoin, it's only available inside our LAN at the moment while I'm preparing an upgrade. Not moin's fault - I had a hard disk go crunch - most annoying - geez, it's a good thing I make regular backups. A few merges, oh, I have to re-do my dark theme too...
Ahhhhh, backups. If there's one "hack" I've taken back out of the computer world into my real life, it's making sure to have backups. Backup plans, backup copies of contact info, someone to backup for me if that old problem about not being able to be in two places at once strikes home. Save early, save often, save extras.
Over in #moin on freenode, where I have been hanging out a lot lately, there's one fellow (deitarion) who's cleaning up not only a lifetime of loose notes but also organizing and commenting on a number of files he's gathered over time, using MoinMoin Desktop Edition, in other words, a wiki designed to be personal: http://moinmoin.wikiwikiweb.de/DesktopEdition
He seems to have saved a bunch of space already and be working on some search engine tricks for it, combining it with GTKtalog (commandline fans may prefer SwissDB ) and planning on a local web proxy to manage his external bookmarks. He's working on saving enough space and/or being clear enough on what's his, to have this help his emigration from his current distro to Gentoo.
[Jimmy] One of the best tips I saw is one from JWZ. He keeps an /etc/setup script, and every time he does anything from the root account, copies and pastes it into the script, and comments it saying why, before copying it to his other machines, so he can have every machine set up to exactly the same state whenever needed.
[Heather] Jim (the Answer Guy) and I both keep a README file for each of our machines, either in /etc or in / (I like to hardlink them together; keeping /etc on a seperate partition from / is rare and creepy, but a symlink would do nicely - hardlinks, though, protect it from being deleted by the fs if one of the links is removed).
This file datestamps any interesting changes we do to a machine, possibly including script bits, fstab tweaks, etc. Reading it should be enough to figure out how to spin up another machine just like this one even without the box itself.
I also keep one of these in each complete chroot area I set up, since I treat them like whole machines. Which brings me to the next Lifehack :) For chroot'd environments that aren't just the single app, I change its /etc/hostname to something that clarifies what it is (past examples include "rescue", "minideb", "dev-potato", "memoryalpha" for my trek stuff), and I change the part in the root user's setup (/root/.bashrc) so that instead of asking for hostname:
export PS1='\h:\w\$ '
...it uses the contents of the hostname file:
export PS1="-=[ `cat /etc/hostname` ]=- \w\$ "
Note the doublequotes - if you use single quotes, the command to cat the file is going to be used every time, and you don't really change a chroot's hostname all the time, do you? (I certainly don't.)
Since my chroot'd prompt also looks different, even without colorization tricks that I use on my terminals, I can easily tell that I'm in the chroot, and which one I'm in. So nice when I'm ssh'd in and using a screen session heavily.
If you haven't heard of screen then for goodness' sake get it. Job control is nice, but this gives whole windows with real terminals, so the human can multitask too; things that you might want to background can be run in another window where they think they have foreground; it can be told to log one of its windows (great when a compile or some other scripted thing is about to spout lots of text you want to read later, or grep through); with some rather careful permission settings it can even be used in multi-user mode, so you can show a junior admin, your kid sister, or a linux newbie you're helping spin up to speed, the exact things that you're doing. With screen you don't need X and vnc unless you need graphics :)
[Jimmy] Hey Raj, you've been holding out! http://del.icio.us/lunatech/lifehacks has the kind of stuff I was looking for and you know it! :-P
[Raj] I never thought anyone else (who is a geek/programmer) would find those useful. I had collected them from manager-type fellows
. You might find these useful too - http://del.icio.us/tag/cli - http://del.icio.us/lunatech/hacks
[Jimmy] What, geeks don't have to deal with managers anymore?
I really liked the "Hack Yourself" link (http://www.bloodletters.com/hackyourself.shtml)
|
............... You don't exist. You just think you do. We're nothing but the stories we tell ourselves. We know in our hearts what kind of people we are, what we're capable of, because we've told ourselves what kind of people we are. You're a carefully-rehearsed list of weaknesses and strengths you've told yourself you have. (Self-confidence, for example, is a particularly nebulous quality you can easily talk yourself out of having.) You owe no allegiance to that self-image if it harms you. If you don't like the story your life has become -- tell yourself a better one. ............... |
The lifehacks tag is a good one to keep an eye on: http://del.icio.us/tag/lifehacks (or even http://del.icio.us/tag/lifehacks+gtd)
- Since lifehacks is pretty much just self-improvement geekified:
- http://c2.com/cgi/wiki?SelfImprovementPatternsRoadMap
"How to Make a Complete Map of Every Thought you Think" (http://speakeasy.org/~lion/nb) is a braindump book about how to keep a paper version of a wiki.
[Heather] I think John Walker's "The Hacker's Diet" counts as his own lifehack: http://www.fourmilab.ch/hackdiet/www/hackdiet.html
[Brian] THANK YOU, Jimmy, for bringing this one up!
[Jimmy] You're welcome :)
|
Contents: |
Submitters, send your News Bytes items in
PLAIN TEXT
format. Other formats may be rejected without reading. You have been
warned! A one- or two-paragraph summary plus URL gets you a better
announcement than an entire press release. Submit items to
bytes@lists.linuxgazette.net
Configuring the GIMP to look and act a bit more like Adobe Photoshop
The Linux Box Show, recorded in glorious audio for your listening pleasure.
Interesting Open Source perspective analysis of CeBIT 2005, and the opportunities missed.
Linux Medical News is 5 years old.
Richard Stallman interviewed on the state of GNU/Linux.
Waiting and Sleeping on the command line.
Linux KernelThe newest iteration of the stable 2.6.x series of Linux Kernels is now available: Linux 2.6.11.6.
Also new, is the April release of version 2.4.30 in the previous stable series.
As always, you should download kernels and patches from your nearest mirror.
Intel LicenseIntel has tentatively proposed eliminating the Intel Open Source License, substituting for it the more common BSD license. This would be a small contribution to reducing the proliferation of licenses that has occurred in recent years.
TUX MagazineTUX Magazine is a new Linux magazine aimed at beginners. Coming from SSC, publishers of Linux Journal, TUX will be a monthly online magazine distributed in PDF format. NewsForge has published a report on the new publication.
Yoper Yoper Linux has been profiled and reviewed at NewsForge.
Ubuntu/UserLinux LWN has reported on the release of experimental UserLinux metapackages for Ubuntu Hoary.
Puppy Puppy Linux is a flexible and small-sized GNU/Linux distribution. You can read a brief review at LinuxInsider.com.
Arch OSnews has published a lengthy interview with the developers behind Arch Linux.
Firefox Firefox appears to be making measurable inroads into Microsoft Internet Explorer's market share. The Microsoft browser offering now accounts for less than 90% of the browser sector.
Nero Nero, the popular Windows CD/DVD burning software, has recently been released for Linux. NeroLinux is a closed source application, and is available free of charge following registration on the Nero website.
Adobe Reader Adobe has released a new version of its PDF reading software, Adobe Reader 7.0, for Linux. The software is available for free download, and has been profiled on DesktopLinux.com.
Mick is LG's News Bytes Editor.
Before this, Michael worked as a lecturer in the Department of
Mechanical Engineering, University College Dublin; the same
institution that awarded him his PhD. The topic of this PhD research
was the use of Lamb waves in nondestructive testing. GNU/Linux has
been very useful in his past work, and Michael has a strong interest
in applying free software solutions to other problems in engineering.
Originally hailing from Ireland, Michael is currently living in Baden,
Switzerland. There he works with ABB Corporate Research as a
Marie-Curie fellow, developing software for the simulation and design
of electrical power-systems equipment.
These images are scaled down to minimize horizontal scrolling. To see a panel in all its clarity, click on it.
All HelpDex cartoons are at Shane's web site, www.shanecollinge.com.
Part computer programmer, part cartoonist, part Mars Bar. At night, he runs
around in a pair of colorful tights fighting criminals. During the day... well,
he just runs around. He eats when he's hungry and sleeps when he's sleepy.
By Adam Engel
The KDE and GNOME desktop environments make it possible for people who have no knowledge of Unix or GNU/Linux commands to use a GNU/Linux system as easily as they'd use Mac or Windows for web-browsing, word-processing, email, games and the things most users spend their computer time on. It's only going to get better, and one day many people might know GNU/Linux only through the GUI.
I asked Okopnik, "Do you think this is a 'good' thing, that people are finding out they have an alternative, but at the same time using that alternative almost exclusively on the GUI level, like they would use Mac or Windows? Have you noticed more of an interest in Linux or an enhanced readership since Linux became both market and user friendly? If so, are these new users less interested in the "technical" aspects than in having a stable GUI-based system to for work and email and net-surfing?"
Okopnik wrote, "Actually, this is an issue that I brought up in an involved discussion with the LG staff and the Answer Gangsters (The Answer Gang answers Linux questions sent to us by our readers, and the discussions and the answers become part of LG.) My viewpoint here is that it's actually a very good thing - modulo the awareness that the Command Line (CLI) exists. That is, people are perfectly welcome to come to Linux and use only its GUI capabilities as long as this serves their needs - but when the GUI proves insufficient, the capabilities of the CLI are there, just underneath, providing the perfect security blanket.
"In an article I wrote for Java Developers Journal, I related an example of this. I had a client whose Web developer left them in the lurch with several hundred HTML files without the ".html" extensions. This wouldn't be too bad by itself - renaming a group of files isn't difficult - but the thousands of HTML links within the files referred to those extensionless names as well. With GUI-only tools, this is a nearly-unsolvable disaster. From the CLI, it was a matter of a single short line of code:
perl -i -wpe 's/<a href="[^"]+/$&.html/g' *
"The readership of LG has certainly changed over time. Where we used to get dozens of questions on fairly technical topics in The Answer Gang, we now get only a few - and they tend to be simpler, less technical. The email I get from our readers indicates that there has indeed been a definite shift in the user base; the old Linuxer who would bang on a problem for hours so that it could be reported (and quickly fixed) is being... well, not _replaced,_ but reduced, percentage-wise, as the mainstay of the population. The new user is often just that - a computer user who just wants that email/web/document/spreadsheet processor and maybe a few games on the side. There is, however, a cultural shift that occurs even in those users after a while: you cannot live in a society based on a given moral premise and ignore that premise, or even stop it from penetrating into your life (even if you try to prevent it.) The original "hacker ethic" of Linux lives on, strong as ever in those who use the full extent of this OS, and inherent (and growing, however slowly) in those who use it even without that full knowledge.
I wrote, "I was considering the license for Documentation -- I forgot what it's called..."
Okopnik wrote, "The Open Publication License. There has been quite an evolution of licenses on the documentation side; the OPL is pretty much the last product of it (there are a few others) and is by far the most popular, simple, and comprehensive. The LG remains under it because I did _a lot_ of applicable research."
I wrote, "It almost goes without saying that this article will be 'free' but it's probably a good idea to say it because I've been 'burned' before, as have other writers I know, having articles that were meant to be "free" stolen by publications that charge a fee to readers -- online or off."
Okopnik replied, "This would be a good reason to implement the 'no commercial distribution without prior permission' clause."
I wrote, "Tell me about your experience with licensing and your research. For instance, how does one particular license affect LG as opposed to another?"
Okopnik wrote, "When I took over the E-i-C/publisher slot here at LG, I decided to revisit the decisions that had been made (and cast as procedure) in the past with regard to as many of them as I could find; in my opinion, this needed to be done to keep LG fresh and relevant. One of these was the license under which we accept and publish all our material.
"I had asked Rick Moen, a member of our staff who is very knowledgeable about the nitty-gritty of various FLOSS issues including licensing to comment on our use of OPL; I also threw the floor open to others' input. As a result of the discussion and familiarization with the basic issues at stake, I did a lot of my own research, and came to focus on the OPL (which was the license LG was using at the time) and the CC, the Creative Commons license (see http://creativecommons.org/; I recommend the site highly, particularly their "license-choosing wizard" which helps you pick the appropriate one for your application.) The latter held a lot of promise and flexibility, but in the end, I had to vote /antiquo/; the OPL, with a little clarification, supplied all our needs. The front page of LG now states:
"All content released under the Open Publication License v1.0 (options A and B not applied)
"The clarification in the parentheses was the only thing added. Given that the author of OPL himself, David Wiley, considers it dated (he became CC's Director of Educational Licenses and shut down the Open Content site in 2003), we may well transition to the CC at some point in the future. For now, though, the OPL serves our purposes and creates no restrictions on the author's rights (theirs supercede ours) that I would find objectionable.
"LG's only concern is the right to distribute freely in electronic format. Should you wish to prohibit or restrict commercial distribution, or want to be contacted before anyone converts it to print, we have no objection or concern; in fact, we have no say - and this is what I, in my capacity at LG, see as perfectly suiting our needs. A slightly different attitude than you'll find in a commercial environment, I grant... " wrote Okopnik
I wrote, "This is a particularly interesting/different attitude for a magazine."
Okopnik wrote, "The Open Source culture often produces those. Once money is not the primary motivator, a number of interesting results show up. FLOSS is a social experiment gone successful -> mainstream -> wild, a meritocracy/gift-based culture that focuses on exchanging people's best abilities for community recognition and respect. Part of the secret of its effectiveness is that you can't cheat people out of respect the way you can with money - it can always be lost or withdrawn. Like any other human system, it has its imperfections (see Cory Doctorow's "Down and Out in the Magic Kingdom" or James P. Hogan's "Voyage into Yesteryear", both of which describe a respect-based system as the basis of an economy), but they're a) usually self-correcting and b) several orders of magnitude less extreme at the edges than what we have now."
I wrote, "It actually brings up the question, "what is a magazine" (online or off)? I always thought of a commercial magazine as a corporation using artists and authors to sell advertisements while at the same time providing a venue for artist/authors' works. But a zine like LG, which provides essential, often crucial information to a specific audience -- i.e. Linux users -- has a unique responsibility as an "education/information venue." By placing author's rights above the rights of the magazine itself, you are serving as a mediator between your audience and the kind of information you offer them."
Okopnik wrote, "/Tetigisti acu/; well done, sir. That is precisely our function and mission. We want to introduce people to Linux, get them to the point where they'll know enough to ask the right questions and to find the answers. This would set off all sorts of alarms for a commercial venture - we are, after all, training our readers not to need us after a while - but I'll happily hang up my editorial hat once most of the world is Linux-competent. :)"
I wrote, "I see LG as a "mediator" between Linux Users and the millions of pages of information pertaining to Linux. At some point an 'editor' is needed, otherwise a reader can just do a Google search on various subjects without a coherent 'theme.'"
Okopnik wrote, "Precisely so. Does it surprise you to learn that I'm a teacher (seminars in programming, network security, etc. for Sun Microsystems and others), and have been, in one form or another, for most of my career? I suspect not."
"What's FLOSS?" I asked.
Okopnik wrote, "An unwieldy compromise of a name that the majority can live with. As you've probably figured from your exchange with Stallman, there's some disagreement about exactly what this whole movement should be called - and Free/Libre/Open Source Software is what we got as a result. Like dead yeast in beer, it's harmless and doesn't even affect the flavor."
I wrote, "Interestingly, or by strange coincidence, I destroyed my wife's Windows installation on a super-powered custom-built PC she uses to run the Maya 3D program (she teaches computer graphics and 3D at New York University)."
Okopnik wrote, "You do realize that Maya is usually run under Linux these days, right?"
I wrote, "Doesn't matter; the university's program is locked into Windows. Anyway, she warned me not to mess up her dedicated Maya machine by 'playing around with Linux' and of course I told her this was an impossibility -- oops. Problem was, I switched, after a year of smooth sailing, from GRUB to LILO and totally corrupted the Master Boot Record (MBR). This brought up an interesting point: virtually no one knows how Windows works. I was able to fix the LILO problem, but I had no idea how to do anything in Windows, nor did many people I called who actually teach courses on the Windows OS. To make matters worse, the only boot disc available was the original installation CD, which you can only access via a password, which we forgot. So, even though we "owned" the system, there was no way of getting in. We had to call in an expert, the person who built the machine and installed the software, the President of Compusoft Computing Systems himself, Philip Keough, who is all of sixteen years old..."
Okopnik wrote, "In an information-based economy, intelligence (and the wisdom to use it correctly) is the winning factor. The side effects still tend to surprise the hell out of people, which makes for interesting food for thought."
I wrote, "I recognized this as an opportunity not only to save my marriage, but to interview a certified computer whiz from the 'younger generation' (painful, those words) who grew up with both Windows and Linux. His high school -- I think the President of Compusoft Computing Systems is a junior -- just provided a Sony VAIO laptop for every student, loaded, of course, with Windows XP. The school was considering GNU/Linux, but decided to go with the 'industry standard.' I asked him if anyone among his hacker friends, not the 'mainstream students,' was into GNU/Linux, which I thought would be the OS for rebellious young geeks. His answer was exaclty 2 out of 2000 sudents, are familiar with GNU/Linux: himself and some other kid who publicly proclaims 'give me Linux or give me death' but secretly runs Windows 2000 as his main installation. I asked Philip why this was so, why kids at such a young age would want to feed into the 'industry standard' way of life. His answer was that Windows felt more "professional" just as Adobe Photoshop was 'more professional' than The GIMP. So much for garage-band rebellion and the Ramones...
"If what he was saying is true, and Mac is viewed as something for graphic artists, like the old SGI, but otherwise obsolete, then there is only one operating system on earth, and it's a damned bad one," I wrote.
Okopnik wrote, "You have to remember that the OS competitive arena is the entire world, while what you heard was a single opinion from a single person in a single location. Furthermore, if he had said 'Windows can do X, Y, and Z whereas Linux can't', or 'the software that we're mandated to use requires Windows', well and good (although I don't think that it's possible to defend the first argument); since his entire area of focus is on how an OS 'feels', then he's off into mysticism as opposed to rational judgement."
I wrote, "So, in reference to your LG audience: is GNU/Linux just a toy that hackers boot on week-ends to unwind, or is it viewed as a genuine alternative to Windows? The feeling I'm getting, or I was getting from Philip, is that even the most dedicated GNU/Linux enthusiasts share their hard-drives with some version of Windows for use in the 'real world.'"
Okopnik wrote, "Well, let's see. Many schools, hospitals, and government installations in India run Linux. Same for South Africa. France has passed a law that says "Open Source is to be implemented whenever possible"; Brazil, which was 8% of Microsoft's business a couple of years ago, has followed suit. Germany has been using Linux in their security departments, and is now implementing it at every level - federal, state, and local. China has decided that it's their official OS; Korea and Japan have joined them in investing several billion dollars in FLOSS software development recently. Most of South America is switching, led by Peru (Dr. Villanueva Nunez, a Congressman, responded succinctly and brilliantly to the fear-and-doubt tactics that Microsoft tried to sow when the decision was made; the translation can be found in many places on the Net.) This is just off the top of my head; there are many other countries which have decided that FLOSS simply _works_ for them (generally by making them competitive in the world market and removing an unnecessary barrier to their pool of talented but poor would-be techies.)
"Special effects - in e.g. Titanic, Shrek, The One, and many other movies - are being done on Linux, simply because you _can't_ build a real server farm for crunching serious processing under Windows (incidentally, one of the most powerful computers in the world is a massively-parallel rig built by students in Australia. Guess what it runs?) The overwhelming majority of Web servers on the Net run Linux - and many of those that don't are running Apache, a piece of FLOSS software.
"I hope these random examples begin to add up to a coherent picture. I couldn't even start to draw an outline of just how huge and pervasive the entire FLOSS penetration into the OS market actually is," Okopnik wrote.
Okopnik added, "Mac's gone FLOSS. Well, almost completely, anyway: the base of Mac OS/X is another free Unix, BSD. The only part they haven't really opened is their desktop manager (if I recall correctly, it's called "Aqua") - but you can run plain ole' X on Mac hardware, and it's available right from Apple's site. What's more, Apple has been edging toward releasing the older versions of Aqua as FLOSS; their real edge has always been as a great hardware and User Interface (UI) company anyway, so they wouldn't lose anything.
"A lot of FLOSS software, unsurprisingly, can be compiled to run on OS/X; the instructions for doing so are, again, available on Apple's site - and so it a lot of already compiled FLOSS. Did I mention that we're taking over the world? :)," Okopnik wrote.
I wrote, "But Mac long ago ceased to be any kind of real competitor to Windows. Like the old Soviet Union kept the U.S. in check and vice versa. Now the U.S. is a Monopoly run rampant -- like Microsoft. A closed system is a closed system. The rest of the world, developing better software on Linux, will catch up while Microsoft, who keeps its position only through use of 'force' will wither up and die," I wrote.
Okopnik wrote, "Is it as obvious to you as it is to me and many others? I don't understand why more people can't see that basic fact - other than simply not knowing or understanding the issues."
I wrote, "I came across this quote by Stallman on the gnu.org site (www.gnu.org/thegnuproject.html):
'The "Linux" magazines ... are filled with advertisements for proprietary software that works with GNU/Linux. When the next Motif or Qt appears, will these magazines warn programmers to stay away from it, or will they run ads for it?'
"Response?" I wrote.
Okopnik wrote, "As it happens, LG is the one Linux publication that does not accept advertisements, and therefore does not follow Stallman's dictum; we prefer to remain totally unbiased (except by our own personal prejudices, of course. :) I have to agree that commercially-driven magazines do have their bottom line to consider; perhaps not above all, but it is a very strong motivating factor. I'll admit to being actually shocked for about 30 seconds when I saw a Microsoft ad in the Linux Journal... but sober consideration prevailed. There's no reason that they shouldn't have - the ad said nothing derogatory about Linux and was placed in the more-or-less correct market for MS's purposes - but it did make for an interesting contrast.
"For the people who were outraged by this - and judging from the comments on Slashdot and elsewhere, many were - I'd suggest considering the following Buddhist koan:
"'Always meditate on whatever provokes resentment.'
"Very interesting questions can be found in the moment between the emotional response and the reaction..." wrote Okopnik.
I wrote, "Sometimes I wonder if I should be writing two different articles, the GNU Story, and Everyone Else's. They're kinda picky with their "GNU/Linux" not "Linux" and initially boycotting KDE because qt was "proprietary" but I guess you need people like that, people who are ready, willing and able to go 'all the way' to get a movement started. If you don't mind me 'requoting' you from the previous section of this article:
'We _need_ our radicals. They're ugly, scruffy, pushy, aggressive, loud, and unfit for normal humans to associate with - but, O Ghod do we need them! They sacrifice themselves on the altar of whatever the hell their passion may be; they give up their right to be seen as "normal", and make of themselves targets at which the majority of society will fling rocks and garbage - and we, the human race, get to move ahead just another tiny notch for each one of them. Granted, there are radicals on either side of the fence - and lots of different fences - but the total vector of these little steps *is* in the direction of progress; another pragmatic belief of mine, and although I won't go into the philosophical ramifications of it, it can be summed up as "'good' is just another way of saying "pro-survival".' -- Ben Okopnik. That's you," I wrote.
Okopnik wrote, "I might add, 'Listening to these folks, however, _does_ require turning down the volume and intensity controls, and keeping a supply of large grains of NaCl handy. :)'"
I wrote, "I just want this article to introduce GNU/Linux to people who aren't aware that it's a valid option -- especially now that KDE and GNOME provide GUI 'desktop environments' that anyone can use. Good to know know LG is sticking to the straight and narrow regarding advertisements for proprietary software, though."
Okopnik wrote, "Well, we don't have an articulated social contract the way Debian Linux does (http://www.debian.org/social_contract), but we do have our priorities. 'Making Linux a little more fun' does not mean distorting the truth for profit - and that includes being cautious with regard to slippery slopes."
This work is licensed under a Creative Commons License. It is free to distribute, reproduce or modify with the author's consent. Read more about licensing software, text and documentation at http://www.creativecommons.org.
Adam Engel's first book of poetry, Oil and Water, was
published by Maximum Capacity Press in 2001. His novel,
Topiary, will be published by Dandelion Books in the
Spring of 2005.
He has worked as a journalist, screenwriter, executive speechwriter,
systems administrator, and editorial consultant, and has taught writing at
New York University, Touro College and the Gotham Writer's Workshop in New
York City.
Adam Engel has published poetry, fiction and essays in such magazines and
periodicals as Counter Punch, Dissident Voice, Online Journal,
Strike-the-Root, LewRockwell.com, The New York Art Review, The Concord
Journal, The Middlesex News, Accent, The Littleton Review, Ark, Smart
Shoes, The Beacon, Literal Latte, Artemis, The Lummox Journal, Fearless,
POESY, The Half Moon Review, Art:Mag, Chronogram, Gnome and others.
Here is a short course (with some excursions) to help you take your existing Desktop, roll it into an ISO pipe and smoke a USB stick or CD. This is not a task for the faint-hearted - there are no automated scripts in here!
Part of the reason for this hands-on approach is that each desktop system will have its own quirks, so writing scripts which handle the various conditions which might arise is painful. Secondly, there are numerous choices possible - you must mix your own. Finally, I must confess that I do not feel energetic enough to write the scripts at this point. Instead, you have this article!
So if you have spent a lot of time fine-tuning your configuration and want to waste some more time putting it onto a (re)movable drive - read on.
Since the chosen medium for our ``live'' portable system is a CD or USB stick we do not want to write to it often. In fact, in the case of a CD-R we can only write to it once. It should be obvious, however, that a ``live'' system does need to write something if it is to be counted as among the living!
While many mechanisms have been suggested to handle this, we
will (essentially) follow the system chosen by bootcd. To do this,
we need to create a directory /wraith, an archive
/wraith.cpio.gz and a script /etc/rcS.d/S01undead.sh.
The contents of the archive are rather system specific so you will need to choose its contents. However, if you are really, really impatient you can download the listing of the archive that I use and run the following:
cd / cat wraith.lst | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gzDid you get a lot of error messages? No? Are you really sure that your system is almost identical to mine? Yes? Then you can skip the next subsection.
How does one find out what files need to be written to on a live system?
One way would be to find all the files that have been written
to on your current desktop. To do this first find out when the system was
booted - a good measure of this is when the root filesystem was last mounted.
So for example
# Use your real root device in place of /dev/root. last_boot_time=$(dumpe2fs -h /dev/root | \ sed -n -e's/Last mount time: * //p')You may wish to use the log files or the
last command instead.
You only need to know the last boot time approximately; subtract a minute
or so from it just to be on the safe side - unless you boot the system
more often than that! Now create a file with that time stamp using the
touch command:
touch -d "$last_boot_time" /tmp/lastbootYou can now create the list of all files that were modified since that last boot (for simplicity we will only bother with the directories
/etc and /var; you can add some other directories if
you so desire):
find /etc /var -newer /tmp/lastboot > /tmp/changed # And, just for fun... find /home -newer /tmp/lastboot > /tmp/home_changedHave a look at these lists but don't delete them just yet. You should notice that there are three types of files that are written to on a running system.
mozilla cache directories.
/tmp/changed, according to this
classification: /tmp/write will consist of those files (mostly
directories) that are empty at boot time but get written to as the system
runs; /tmp/links will consist of the files that will be
quasi-static - we will keep a static version of these files at boot time
but we might want to change them on a running system. We will include the
third category of files in /tmp/links as well, since we will not
treat them differently - but ultimately you may want to change this.
We first create a directory to hold the files that will be modifiable
at run-time - say /wraith. Mount a RAM-based file system on it
by mount -t tmpfs tmpfs /wraith. Big Fat
Warning: This file system is ephemeral and will be lost
when you halt the system. If you wish, you can use the directory as-is
(without the tmpfs mount) during this subsection, but don't
forget to clean up its contents once you have created the archive as
explained below.
In /wraith, we will create the top-level directories like
etc, var, tmp and so on that we will
want to write to. In these directories we will create the files as per the
classification above. First, we'll do the writable but empty files:
cd / cat /tmp/write | cpio -pdum /wraithWe expect these files to be empty at start-up so we will ``zero'' them. Do this only to the files for which you don't want to keep the contents. For now I assume these are all the files in the list
/tmp/write:
for file in $(cat /tmp/write) do if [ -f $file ] then > /wraith/$file fi doneOf course, we also need an empty
tmp directory:
mkdir /wraith/tmp chmod 1777 /wraith/tmpNext, we create the links.
cd / for i in $(cat /tmp/links) do dir=$(dirname $i) top=$(echo $dir | cut -f2 -d'/') rest=$(echo $dir | cut -f3- -d'/') name=$(basename $i) mkdir -p /wraith/$dir ln -s /wraith/$top.ro/$rest/$name /wraith/$i done # As a safety measure to ensure that all configuration files # have been created mkdir -p /wraith/etc cd /etc for i in * do ln -s /wraith/etc.ro/$i /wraith/etc/$i doneThis is more complicated and needs further explanation. The idea is to make the ``static'' versions of the files available under the
.ro
top-level directories. So, for example /wraith/etc/hostname will be
a link to /wraith/etc.ro/hostname.
To see this at work create etc.ro and var.ro as
sub-directories in /wraith. For each of these directories (say etc)
we run a pair of commands like the following. (Warning: Be careful here. If
you haven't created all the links in /wraith/etc as above you may crash
your running system).
mount --bind /etc /wraith/etc.ro mount --bind /wraith/etc /etcAfter these mounts, the file
/etc/hostname is a link to
the original hostname file which is now available as
/wraith/etc.ro/hostname. Since the left-half of this
link is on the RAM disk we can perform replacement surgery on it:
vi /etc/hostname.new mv /etc/hostname.new /etc/hostnameOn the other hand, if you want to change a file in a sub-directory of
/etc, it's a little more complicated:
mkdir /etc/X11.new ln -s /ram/etc.ro/etc/X11/* /etc/X11.new mv /etc/X11.new /etc/X11After this you can edit the files in
/etc/X11. Yes, this is quite
twisted but (once you get the hang of it) not difficult to
manage - especially since we expect that we will edit these files
only rarely. An alternate approach is to create the directory tree under
/etc in its entirety only leaving links to the files.
You can use the above mounts to test your choices of
/tmp/links and /tmp/write, but the real test will
come later. For now, undo the above mounts by a pair of commands like:
umount /etc umount /wraith/etc.roYou can also remove the
.ro directories if you like.
Finally, we create an archive of this directory:
cd /wraith find . -xdev | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gzThe
cpio command will tell you how many 512-byte blocks you
wrote. If the archive is really large (more than 1MB or so) then you
probably need to re-do your choices.
We need a mechanism to bring the /wraith directory into
operation at boot time. To do this, install a script like the following
one to run early at boot time. For example you could install the script as
/etc/rcS.d/S01undead.sh.
# # undead.sh Mount and load up the /wraith directory for use # # Version: 0.3 01-Feb-2005 # # If this has already been run then don't run it again. # We can't handle two wraiths! [ -f /wraith/live ] && exit 0 # Create writable space mount -n -t tmpfs tmpfs /wraith # Create the directory structure cd /wraith gzip -dc /wraith.cpio.gz | cpio -idum cd / # Perform the cross mounts with bind # which is like a directory hard link. cd /wraith for i in * do mkdir $i.ro # We use mount with the -n # To avoid confusing the mtab mount -n --bind /$i /wraith/$i.ro mount -n --bind /wraith/$i /$i done cd / touch /wraith/live : exitFinally, you edit
/etc/fstab so that the root filesystem is
mounted read-only at the next boot - just change defaults to
read ro,defaults in the appropriate entry.
Reboot and that's it! You have a read-only root system... or
almost. Actually, it is likely that you will find a number of places where
you didn't create the links you need or created the wrong links. Don't
worry. You can modify the /wraith.cpio.gz archive to your heart's content.
Make the changes you need to the ``live'' /wraith directory.
Now copy all the changes from /wraith into /tmp/ghost. The command
cd /wraith find . -xdev | grep -E -v '^./((live)|([^/]*\.ro))' > /tmp/listwill generate the newer list of files. You can unpack the older archive and compare its list of files with
/tmp/list.
mkdir /tmp/ghost cd /tmp/ghost zcat /wraith.cpio.gz | cpio -idum find . -xdev > /tmp/oldlist wdiff -a /tmp/list /tmp/oldlistUsing the differences you can see what files you need to create in
/tmp/ghost in order to match it up with the running
/wraith. You can save your changes by something like
cd /tmp/ghost find . -xdev | cpio -o -H crc | gzip -c -9 > /tmp/wraith.cpio.gz mv /tmp/wraith.cpio.gz /wraith.cpio.gzThe changes will become automatic at the next boot. Of course, once you write the filesystem to a CD, you will have no chance to revise it again!
All this looks too complicated and life is too short? Just remove the
script /etc/rcS.d/S01undead.sh, the archive /wraith.cpio.gz
and the directory /wraith. You will have your system as pristine
as before.
We want our system to ``run anywhere'' - in particular, we should be
able to mount our root file system whether it resides on a CD or USB
stick (or perhaps even hard disk). If we use a CD then on
most systems this will be on the device /dev/hdb or /dev/hdc.
The USB stick usually shows up as /dev/sda or
/dev/sdb. It should be relatively simple to just
create a kernel which supports IDE CD drives and USB block devices.
When we boot such a kernel (with the correct root=<device>
parameter) the system will start up as expected on 90% of all systems
that one is likely to encounter. If this is OK with you then you
don't need an initrd so skip the rest of this section and read
the
HOWTO
on building the kernel with IDE CD and US support - don't forget
support for the ISO 9660 (CD), ext2 and vfat (Win95) file systems.
What about the remaining 10%? That will take 90% of the work as usual. One possible solution could be to build all the drivers of all possible CD drives, USB readers and the like into the kernel. Unfortunately, automatically probing for some of these devices will occasionally cause other devices to choke-up. It also seems like a bit of a waste to take up such enormous amounts of kernel memory for unused drivers. The solution provided by our intrepid kernel hackers is the modules+initrd mechanism which allows you to write a script that chooses which drivers to load depending on the devices found.
The boot loader (see the next section) will load the kernel and the
initrd into memory. We will use a ``standard''
Debian kernel image
which has essentially everything modularized (``essentially'' since we
must have support for at least one file-system built into the kernel
in order to load the init RAM disk - this could change if I understand
initramfs better).
After the kernel has done its thing, it sets up the file-system with root
on the initrd and executes /linuxrc but doesn't quite fully let
go - when /linuxrc exits, the kernel executes /sbin/init.
We follow Debian's choices when we visualize the boot process as follows:
/sbin/init (still on the initrd).
/sbin/init on the initrd is a script
that will run the following scripts:
mount_root
to recognise and mount our chosen file system on /mnt.
init are
/mnt and cd
to it.
pivot_root which makes the current directory the
root directory for the kernel and mounts the old root directory at
/initrd. After this our ``real'' root file-system is
mounted as root.
chroot . to change the root device of the
current process so that /initrd is free to be unmounted. We
must do this so that the RAM disk is free to be unmounted which frees
its space for use by other processes.
/sbin/init on the real
root file system. This is the ``real'' init program which will
initialise the live system.
/linuxrc and /sbin/init needed for the initrd. So we
only need to provide the scripts loadmodules and
script.
Writing these scripts was one of the most complex steps for me as it deals with the aspect of Linux that I usually encounter the least - at least on a working system - booting! On the other hand, this is the job for which most installers and other forms of pre-install detection tools have been written. So we follow the ``teacher'' Tom Lehrer's dictum:
Plagiarize, Let no one else's work evade your eyes, Remember why the good Lord made your eyes, So don't shade your eyes, But plagiarize, plagiarize, plagiarize... (Only be sure always to call it please, "research".) -- Tom Lehrer, "Lobachevsky"
There is a good IDE driver detection routine that is part of the standard
Debian initrd. The Knoppix initrd gives us a safe order in which to load
all the SCSI modules. The Linux-Live initrd has a list of the necessary USB
modules to boot off a stick. So we put all these together to get routines
which I call loadmodules_ide, loadmodules_scsi and
loadmodules_usb. The loadmodules script on the initrd will then
act as a dispatcher - it will choose which routine to run depending
on what boot time parameters we give.
Still, we should do some work. So (plagiarising from the
hotplug scripts)
I also wrote a procedure loadmodules_pci
that loads only those modules which correspond to devices
in /sys/bus/pci/devices which match the data found in
/lib/modules/kernel-version/modules.pcimap. This procedure
makes use of the sysfs file system that was introduced with Linux
2.6.x but something similar may be possible using /proc/bus/pci in
Linux version 2.4.x. The principle is that the kernel does
provide a list of all the PCI devices that it found; for each such
device it also provides some device information - the interface for
this is the sysfs file system or (in 2.4.x) the proc file system.
On the other hand, each module writer makes a list of all devices that
the driver is known/expected to work for - the kernel build process writes
these to modules.pcimap. By matching the two lists we should be
able to load only those modules which have a matching device. This only
works with PCI devices but most devices on PC's nowadays
(including SCSI cards and the USB controller) are PCI devices.
Here is the script to loadmodules that resulted from the above deliberations. This scripts depends on a list of modules that are related to block devices.
The second script we will use
provides the routine to mount the root device. Again the sysfs
file system provided by the 2.6.x Linux kernel comes to the rescue. Under
/sys/block we find a list of all the block devices on the
system. If the root= option is given to the kernel we can
check whether this block device is available. Otherwise we check each
available block device to find evidence that it is our root file system -
by checking for the existence of the archive, directory and script that we
created above.
The Debian initrd-tools package is
a collection of scripts and so can be installed on any GNU/linux system
(for example use the source package directly). The main script is
mkinitrd which will create the standard Debian initrd. We will
run this script and make some changes in order to create our ``special''
initrd. First off all create some directory say
/tmp/mkinitrd.confdir. In this directory we will create the
file exe containing the list of executables that we want in
addition to the ``standard'' ones like /bin/sh - in our case
we need /bin/grep. Next we create a list of all the additional
files that we want to include; this is mainly the list of all modules that
are in some way connected with the use of block devices; here is my list. Finally, we also need a configuration file. We are set to
use mkinitrd with this directory as our configuration
directory.
mkinitrd -r "" -k -d /tmp/mkinitrd.confdir -o /dev/nullThis will tell you the name of the working directory which will be something like
/tmp/mkinitrd.1234. Now you need to edit the
/tmp/mkinitrd.1234/initrd/linuxrc.conf file to reflect the
various file systems that you may use for your root file system.
Finally, you copy the scripts you created above and generate the
initrd with mkcramfs.
dir=/tmp/mkinitrd.1234 rm $dir/initrd/scripts/* cp allmod.list $dir/initrd/etc cp loadmodules $dir/initrd cp script $dir/initrd chmod +x $dir/initrd/loadmodules chmod +x $dir/initrd/script mkcramfs $dir/initrd initrd.imgIf you build a kernel with
ext2 filesystem support instead of
cramfs, then you need to create an ext2 filesystem
image based on the /tmp/mkinitrd.1234 directory instead.
We now combine the ideas of the previous two sections. I assume that you have managed to make your root filesystem boot in a ``read-only'' mode and that you are currently running in that mode. I also assume that you have created an initrd that can boot on ``any'' machine.
I know that the latter requirement is hard to check given that you have access to only one machine at a time. Moreover, it is difficult to find friends who will agree if you say ``I have on this floppy an initrd and kernel that I would like to test on your system''; those few will not remain friends if your kernel+initrd manages to fry their system.
In order to boot off a CD or USB stick we need some software that can
do that. The nominees are isolinux,
loadlin and grub... and the winner,
in this case, is... grub.
The main file for booting using grub is called stage2 or
stage2_eltorito in the case of booting CD's. When these files
are properly installed (see below how this is done for CD's), they are
loaded and run by the booting machine. They look for a configuration
file /boot/grub/menu.lst. We use a menu.lst file that
looks like:
default 0Other than the kernel and the initrd, we needtimeout 5
# Pretty colours color cyan/blue white/blue
title Debian GNU/Linux with myinitrd root (cd) kernel /boot/vmlinuz-2.6.8-2-686 root=auto ro quiet vga=791 initrd /boot/initrd.img boot
stage2 and
menu.lst in order to complete the list of steps given at the
beginning.
First you need a ``pristine'' copy of the root file system.
If you used the bind mounts procedure to make the root
file-system read-only, then you can just do
mkdir /tmp/pristine mount --bind / /tmp/pristineYou then make a compressed tree of this file system:
mkzftree -x /tmp/pristine /hugeroomwhere
/hugeroom is some place with a lot of disk space.
Remove the directories /hugeroom/lost+found and
/hugeroom/boot from under this directory. Create an empty
/hugeroom/boot directory to which we copy the kernel image and
initrd. Into the /hugeroom/boot/grub directory goes the file
stage2_eltorito along with menu.lst. These files
will not be compressed.
We now create the CD image:
mkisofs -R -J -z -hide-rr-moved -cache-inodes \ -b boot/grub/stage2_eltorito -b boot/boot.cat \ -boot-info-table -boot-load-size 32 \ -no-emul-boot -o mylivecd.iso /hugeroomThen we blank a CD (if necessary) and write our image to it. For a USB stick, we just create a partition and dump the entire image to this partition using
dd. Since I do not have a system that can
boot off a USB, I can only check the floppy based boot for such a
system. Perhaps one of the readers can enlighten me on how this is to
be handled for USB-booting BIOSes.
You'll probably want to add a writable /home directory to
your system. You need to further customise wraith.cpio.gz for
that. Another thing that you probably want to do is to fix the
/etc/fstab file that goes onto the CD. Other config files may
also need to be customised; /etc/X11/XF86config-4 comes to
mind - for this to work ``anywhere'' it is best to use the
vesa driver. Similarly, use dhcp to configure
ethernet rather than a hard-coded IP address in
/etc/network/interfaces. On most systems there is a
hard disk and it is shame not to use it. You can set-up a swap partition
after you boot from the CD - be careful not to trash the host machine
though!
Today one can find a number of GNU/Linux systems that work off Live
CD's. There is Knoppix - and then
there are its Klones.
There is LNX-BBC, tomsrtbt, LTSP and even one called Puppy! There are the CD-based
installers for the common distributions. But, I am still not
satisfied. Each of these make choices that I am not comfortable with. They
choose KDE, Gnome or fluxbox, when what I want is
fvwm; or they choose xmms when what I want is
alsaplayer (in daemon mode)... and so on.
What's wrong with Sunil's excellent
article then? - just take a minimal Knoppix-like DSL and re-master it.
I would object that Knoppix puts everything in a cloop image
which makes it difficult to read the ``real'' contents of the CD on a
generic system; further this also makes it difficult to master and/or
re-master.
There are other approaches like that taken by Gibraltar, bootcd or dfsbuild or linux-live.
The first two keep the files in a compressed ISO file-system. That makes it usable ``anywhere''. I did try these but for one reason or another they didn't work for me. For example they required the installation of additional packages on my desktop.
Ultimately, it comes down to this: I'm a terribly nit-picky kind of person, and I have spent a lot of time fine-tuning my system and no one is allowed to dictate what packages I must install and how they must be configured.
[ I like this Kapil guy, and the way he thinks. :) -- Ben ]
I enjoy tinkering with such things, and so I must have a system that I understand fully. People also mentioned additional kernel features in late 2.4.x and early 2.6.x that simplify the building of a live CD. Finally, isn't it fun to ``roll your own''?This document was translated from the LaTeX Source by HEVEA.
Kapil Hari Paranjape has been a ``hack''-er since his punch-card days.
Specifically, this means that he has never written a ``real'' program.
He has merely tinkered with programs written by others. After playing
with Minix in 1990-91 he thought of writing his first program---a
``genuine'' *nix kernel for the x86 class of machines. Luckily for him a
certain L. Torvalds got there first---thereby saving him the trouble
(once again) of actually writing code. In eternal gratitude he has spent
a lot of time tinkering with and promoting Linux and GNU since those
days---much to the dismay of many around him who think he should
concentrate on mathematical research---which is his paying job. The
interplay between actual running programs, what can be computed in
principle and what can be shown to exist continues to fascinate him.
It must be the former high school English teacher in me that hears the omission of the subjunctive in movies, TV ads, and TV programs.
No, it is not a tense like the present, past or future tenses. No, it is not a part of speech like an adjective, nor is it a verb form like the infinitive. The subjunctive indicates a mood similar to the indicative mood phrase, which indicates a situation of fact, e.g. a phrase such as "The wind is blowing today". Subjunctive is also similar to the interrogative mood phrase: e.g. "Did you wind the clock today?"
The subjunctive mood can also be used to suggest a situation contrary to fact: e.g. "If I were rich I would buy a new car" or: "I wish I had enough money to buy a new car". The flags in English that indicate the subjunctive mood are the words "if" and "wish".
English is very simple compared to other languages such as French, Spanish, German, Italian, etc. All have several pages in their grammar textbooks devoted to the conjugations needed for the subjunctive mood. No wonder English is becoming the common language, the lingua franca, of the world! The basic reason, of course, is the simplicity of its grammar. Even most scientific or technical papers are in English (just a hundred years ago, they would have been written in Latin.)
English, though, does have its idiosyncrasies which are difficult for foreigners (as well as for voice recognition software) to cope with - including spelling and associated pronunciation. Take, for example, the word spelled "wind". Depending on the context of the sentence, we can pronounce it two different ways - such as in "Was the clock wound?" versus "Did the wind of the blowing wind wound your ears?"
There are a myriad of other words with the same dual pronunciations: e.g. wound, refuse, compact, and contract. There is no end to them! And then there are the irregular verbs - 300 of them - which have very complex tense structures. My husband and care giver (I am a disabled woman in a wheelchair), who came to this country after the Hungarian student revolution was crushed by the Soviets, became one of the original geeks and never really learned the irregular verb tenses. Instead, he uses the auxiliaries, which he learned early and used in phrases such as: " I did in fact see it" instead of "I saw it". It's no surprise that the voice recognition systems prefer such expressions, which are much easier to parse.
I have taught "English for Foreigners" for 20 years (my wheelchair existence did not affect that, in fact it may have even helped) to students on all levels, from beginners to highly advanced technical experts. My husband is my worst student, but he does act as my voice recognition robot sometimes instead of my computer - though he does talk back and the computer does not.
I must say, I am impressed that so many foreigners speak, understand, read, and write English even better than many Americans. I am impressed, indeed, although I recognize they have no choice if they want to succeed at some higher level of occupation. The Internet, the now famous Web, has cemented this fact. Almost all websites - (addressed by www... and ending in either dotcom (in the form of ".com") or dotgov or dotedu or dotorg, etc.) are published on the web in English. Even the website addresses (called URLs, Uniform Resource Locators) end in dotde (Germany) or dotjp (Japan) or dotit (Italy), and symbols for all the other 290-plus similar names of the countries of the world, are published in English.
Google, the most favored search engine on the Web, indicates that at this time there are 1,900K dotcom, 105K dotgov, 370K dotorg, 158K dotedu and 385K dotnet webpages on the Internet (K, of course, stands for kilo which is one thousand). But, most interestingly, the German-based dotde accounts for 2,050K webpages, and more than half of those are in English.
The Web has had a profound effect on English. In place of proper Webster-style English, a new language has evolved on the Internet: "Weblish" (if I may be so bold in naming it). This new language is now used all over the world, and it is constantly coming up with new words most of which have an "e" or an "i" up front. Email is, of course, a well known example, but once email was created, there had to be a new name for the regular mail - so "snail mail" came into use as a great descriptive term.
Examples are all over the Web, and more are cropping up every minute. Just look at the URLs of the website themselves, and you will see a whole new set of descriptive names: Froogle, the name for the shopping website of Google, is one such imaginative Weblish variation of English.
One of my favorite new words is Lindows, the Linux based variation of Windows. We all hope it will succeed.
But there is another interesting development on the Web. As part of Weblish, there are the new hieroglyphs (also known as ASCII art) which many people now use in their emails. For example, I am a disabled woman in a wheelchair, so I like to sign my email with the following combination of letters and characters that looks like a wheelchair symbol (click here to skip the ASCII text):
~\__ (o)\_There are many other examples of this, like :-) for happy and so on.
My aging geek husband immediately suggested that we should call this new symbology eglyphy or webglyphy or... and he went on interminably with new words until I screamed: "Please spare me, enough of this!"
But he is right, somebody will come up with the best name - and it will be adopted by many and become part of the new language.
Then there is SMS, the new cellular phone based Short Message Service. It uses some of the best shorthand out of necessity, since phones have only 12 keys. A good example: "CMI L8R" for "call me later". And so on - you get the drift.
But the most important development of simplified, short word-based English is due to voice recognition. Hands-busy, eyes-busy people, as well as those with a functional disability can benefit greatly from voice recognition because they don't have to use a mouse and keyboard to document their findings.
Voice-activated, easily-used telephone systems will benefit people in all walks of life. Anyone driving a car will find voice recognition a much more effective way of manipulating a vehicle and communicating from the vehicle. We are all hands-busy or eyes-busy at one time or another - in the kitchen, in the garden, or giving care to children or adults in need, etc. Personal computers have the capacity to accommodate voice recognition systems, like IBM's ViaVoice. This is especially advantageous to a large population segment of people with disabilities, those who are chronically ill, and older people as well as their caregivers.
Using a keyboard is next to impossible or at least difficult for this fast-growing group of people. Caregivers and their patients would benefit from being able to use just their voices to document the treatments or care they provide to their patients. Additionally, voice recognition technology would allow them a hands-free environment in which to analyze, treat, and write about particular cases easily and quickly.
The care-giving services market alone may justify the Linux-based voice recognition project. Providing care to the needy is one of the largest expenses in the Group of Ten nations, and it is the fastest growing sector as well. Just in the USA, the segment of the population which includes older people and people with disabilities and/or chronic illnesses accounts for 100 million people. Add to that the 5 million formal caregivers and 44 million informal care givers at work in America, and we are looking at half the population!
In the care-giving field, the simplicity, reliability, and low cost of Linux for servers, tablets, embedded devices, and desktops are paramount features. Obviously, the market for these new technologies exists. What remains is for some courageous company with aggressive people to tap into that market. Once those companies get the technology distributed, the needs of many will be met, and a new mass market will open up - one that isn't currently being filled. In fact, the field of opportunity already exists, but it needs to be expanded to serve people with physical and functional disabilities.
Yes, voice recognition offers great promise for the future. However, it isn't perfect and it still needs to be improved. One improvement would use lip reading to bolster its accuracy. Still another would be multi-tonal voice input. Another would be improvements in the design of directional microphones. Every generation of voice recognition software will be improved as the hardware for Linux gets bigger and stronger.
IBM is, in fact, now working on a lip reading system for installation in an automobile so that the on-board computer can very precisely understand the spoken commands of the driver. So, IBM, we need to get this technology into our desktops, ebook readers, tablet computers, and PDAs as well - please? Why not license your lip reading technology to a consortium of open source developers, perhaps sponsored by a non-profit such as DRAIL, an organization concerned with the needy? Or, even more effective would be a licence for a group of such organizations, such as Robert Wood, AARP, Elderweb, etc. who would put lip reading and enhanced, precise voice recognition on an ebook reader, such as www.ebook-gemstar.com or the Korean manufactured one: www.ebookad.com/hiebook/) or an iPOD (Apple's great music machine), or a tablet computer, or any other PC that this segment of the population would feel comfortable with.
Virtual PCs running on a community-based server to which the end-users could connect with simple telephones (POTS) or video telephones (e.g., www.vialta.com) would be of great use. My geek caregiver husband suggests that we should call this project Wordows. "Oh no", I yell at my favorite geek, "That is not so good: it sounds weirdo!" You need to be careful when you create a new word in Weblish. So let us call it, if you will permit me, "Slimdows".
With all these changes, the English teacher in me comes to terms with a changing linguistic environment. Since life itself is constant change, English must also change.
Janine has been a user of voice recognition software for several years and is an advocate for Open Source solutions that will benefit people with disabilities, such as the Linux-based voice recognition project.
I am a registered principal investigator with NSF.
I am a disabled woman, but have great experience with voice recognition
and have an advanced system with IBM Via-Voice, so my writing abilities are
not impaired at all.
I have many years of personal experience using assistive technologies
(AT), and find it very helpful in SPMS (secondary progressive multiple
sclerosis). In spite of my handicap, I find it gratifying and fulfilling to
concentrate my efforts on worthwhile projects for very deserving companies.
Involvement such as this has proved to have healing power for me: I am
living proof of the power of behavioural medicine based on deep personal
involvement. The best way to use these intellectual strengths is to get
involved with collaborative teamwork and personal communications within the
disabled community and with companies who provide assistive technologies
for this community.
As a handicapped woman who still has control of her mental faculties and
voice, I have something to offer by connecting the right people so that I
can integrate systems through the Internet to develop a mutually beneficial
virtual community.
Personal communications and collaborative teams need assistive technologies
to further the self-esteem of the disabled. In short, involvement allows
disabled people not only a distraction from their problems, but also a
constructive way to spend their time while helping a cause they believe in.
Janine M. Lodato
~|__
( o )\_
By Ben Okopnik
Originally published in Issue 54 of Linux Gazette, May
2000
Never write it in 'C' if you can do it in 'awk'; Never do it in 'awk' if 'sed' can handle it; Never use 'sed' when 'tr' can do the job; Never invoke 'tr' when 'cat' is sufficient; Avoid using 'cat' whenever possible. -- Taylor's Laws of Programming
Last month, we looked at loops and conditional execution. This time around, we'll look at a few of the simpler "external" tools (i.e., GNU utilities) that are commonly used in shell scripts. Recall that shell scripts are made up of 1) internal shell commands and structures, 2) external tools, comprised of the standard utilities, and 3) installed programs; the first are always going to be there, as long as you're executing the script with the same shell (the shebang usually takes care of that), the second will usually be there (but watch out for non-portable syntax between different versions, e.g., the '-A' switch in 'cat', the various levels of regex parsing in different 'grep' versions, etc.), and the last is essentially arbitrary since you don't know what another person executing your script has installed (or not installed) on their machine. If you're planning on distributing your script, you may need to write code to test for the presence of any external programs you use and issue warnings if they're absent.
Oh, and - the reason for the above quote: the tools available to you as a script writer, as you might have guessed from it, are arranged in a rough sort of a "power hierarchy". It's important to remember this if you find yourself continually being frustrated by the limitations of a specific tool, it may not have enough "juice" to do the job. Conversely, it does not make sense to use some heavy-duty utility that's big and slow to perform a simple operation.
Some years ago, while writing a script that processed Clipper database files, I found myself pushed up against the wall by the limitations of arrays in "bash"; after a day and a half of fighting it, I swore a bitter oath, glued a "screw it" label over the original attempt, and rewrote the entire thing in "awk".
It took a total of 15 minutes.
I didn't tell anyone at the time; even my good friends would have taken a Clue-By-4 to my head to make sure that the lesson stuck...
Don't be stubborn about changing tools when the original one proves under-powered.
Strange as it may seem, 'cat' - which you've probably used on innumerable occasions - can do a number of useful things beyond simple concatenation and printing to the screen. As an example, 'cat -v file.txt' will print the contents of "file.txt" to the screen - and will also show you all the non-text characters that might normally be invisible (this excludes the standard textfile characters such as `end-of-line' and `tab'), in '^' (for Ctrl- characters) and 'M-' (for Alt- characters) notation. This can be very useful when you've got something that is supposed to be a text file, but various utilities keep failing to process it and give errors like "This is a binary file!". This capability can also come in handy when converting files from one type to another (see the section on 'tr'). If you decide you'd like to see all the characters in the file, the `-A' switch will fill the bill - `$' signs will show the end-of-lines (the buck stops here?), and `^I' will show the tabs. Note that '-A' is just a shortcut for '-vet' - something that used to be known as "taking your cat to the vet". (Yes, Unix command usage can be quite odd. :)
'-n' is another useful option. This one will number all the lines (you can use `-b' to number only the non-blank lines) of a file - very useful when you want to create a `line selector', i.e., whenever you want to have a "handle" for a specific line which you would then pass to another utility, say, 'sed' (which works well with line numbers).
'cat' can also be used for "here-doc"s - i.e., to gen