Contact Me!!!

spiders

down to end and the site search box

 

Archiving your Yahoogroups to MBX files

Running Yahoo2Mbox | Yahoo2Mbox Syntax | The Use of "--User" and "--Pass" | The Use of "--Delay" and Multiple downloads

http://www.tt-solutions.com/en/products/yahoo2mbox/ provides the official line these are my jottings

If you run a YahooGroup and would like to save a local copy of the messages and/or are well p*ssed off with the fact you can only search 100 messages at a time via the Y!G portal, I might have the answer for you...

This package collates the messages sent to your list as an MBX file....

"A what?"...

Take a look in your mailpackages folder where it keeps your messages... if you are lucky you will see MBX files

Now the package I am going to describe generates MBX files from your yahoogroups list posts so you can open and search them like your own emails! Eudora is pretty forgiving when you plop a MBX file into your eudora folder next to all the others.....

You can even convert MBX files to HTML using gee whizz things like Hypermail (I'm really scared here as I'm gonna need to run UNIX programs on my PC via a LINUX emulator called Cygwin!)

These instructions to use yahoo2mbox are for idiots like me who work in WINDOWS

Step one: download and install ActivePerl from Active State: here

I installed it in a subdirectory on the drive housing my windows drive called d:\perl

The installation package should add "perl" as a recognised program name too!

This will enable you to run PERL scripts on your own machine ( perl scripts? you know those things that run your webforms etc etc)

WHY do you need this?

Because REAL PROGRAMMERS use CGI scripts to do things and their machines run UNIX and PERL!

Step two: download yahoo2mbox.pl fromTT Solutions: here or somewhere on the web

Save it into somewhere like c:\Perl\

Active Perl is now waiting to run a program that was destined to run on a server, on a real computer!

There are online instructions for using yahoo2mbox here however I will give you some tips of my own here

Running yahoo2mbox | Top

First you need to make sure the program is OK to run using ActivePerl

.....basically rip out the line at the top of yahoo2mobx.pl that says where perl is!!!

ActivePerl will be running the program not perl on a server!

You can either run the package from a "DOS" command prompt hit "start" then "run" then "command" or "cmd" then type a command in the DOS style window

OR For a less stressy option:

open wordpad / type the yahoo2mbox command you like...... / save the file as something.bat and then when you want to run it double click on its icon on your desktop

That WILL run it BUT the DOS window it runs in will disappear if yahoo2mbx falls over, if you enter a bad command via the command prompt you will at least get error message(s) lke this

A half-way-house is to write the command in notepad, copy it, place your mouse at the command prompt and right click... that pastes the test into the command prompt... then hit <return>

I personally prefer to use batch files, especially if you are updating archives of whole lists at regular intervals

These BAT files can live anywhere so long as the file locations are stated ok.. your PC knows where perl is and you tell it where yahoo2mbox is too!

So here is an example, I would call this file fredlist_0_to_1500.bat and it will save the first 1500 messages of a list called fredlist in an MBX file located in the c:\perl\fredlist\ directory running yahoo2mbox.pl found in the c:\perl directory

perl c:\perl\yahoo2mbox.pl --x-yahoo --resume --delay=10 --end=1500 -o c:\perl\fredlist\fredlist.mbx FredList

notice I use a few extra variables AND I give the list name JUST AS IT APPEARS AT YAHOOGROUPS

Fredlist or fredlist instead of FredList could cause problems

Also notice "--" prefixes all the options except o which is -o

"--x-yahoo" sticks a message number in the retrieved message to cross reference the post

Work out what each part of the command above does from this reproduction of the help file from Version: 0.17 Created: June 25, 2001

Yahoo2Mbox Syntax | Top

$progname [options] [-o <mbox>] <groupname>

Retrieves all messages in the archive of the given group from Yahoo and stores them in the specified local MBOX file. If the file exists, the messages not already in the file are appended to it.
--help give the usage message showing the program options
--version show the program version and exit
--verbose give verbose informational messages (default)
--quiet be silent, only error messages are given
-o mbox save the message to mbox instead of file named groupname
--start=n start retrieving messages at index n instead of 1
--end=n stop retrieving messages at index n instead of the last one
--last=n retrieve last n messages from the list
--resume resumes previously interrupted downloading, needed if --start is used, otherwise is automatic if mailbox already exists
--noresume don't resume, **overwrites** the existing output file if any
--user=name login to eGroups using this username (default: guest login)
--pass=pass the password to use for login (default: none)
--cookies=xxx file to use to store cookies (default: none, 'netscape' uses netscape cookies file).
--proxy=url use the given proxy, if 'no' don't use proxy at all (even not the environment variable http_proxy which is used by default),may use http://username:password\@full.host.name/ notation
--country=xx use the given country code in order to access localized yahoogroups (default: none; supported: ar,br,de,dk,es,fr,it,mx)
--x-yahoo add X-Yahoo-Message-Num header to the downloaded messages
--delay=n sleep for n seconds between each message request (may be used allow the script to run continuously without hitting theYahoo bytes/hour download limit)

If you have difficulties, check for missing "--"'s first

You can make batch files to (for example) grab the next 7500 postings from post 22501...

perl c:\perl\yahoo2mbox.pl --x-yahoo --start=22501 --end=30000 --resume --delay=10 -o c:\perl\mylist_archive_22501_to_30000.mbx mylist

Use of "--User" and "--Pass" | Top

If you are the listowner, then it is fine to have copies of all the posts from your list with complte emails, HOWEVER if you list does not dispay these to members then DO NOT USE "--User" or "--Pass" unless the sign in give a "munged" address like fred@...... If you are a moderator or listowner, signing in with your user name and password may possibly result in full disclosure of the email addresses in the messages?

If your group is set only to give access to archives if you are logged in, you MUST use "--User" and "--Pass" in order for Yahoo2Mbox to work!

Use of "--delay" and multiple downloads | Top

Yahoo allows you only so much bandwidth, if you download too much too fast you will get a scrolling window confirming multiple download failures.

If this occurs when you are only running ONE download, increase (or use) "--delay". i have found a delay of 10 with a 512k ADSL connection will support 4 simultanous downloads of TEXT ONLY lists (see below) So long as you see read "xxxx bytes" you can be sure the downloads are successful.

If your list allows attachments you are in big trouble, expect to have to play with "--delay"

Problems?: Rapidly scrolling screens with no sign of "bytes" indicate a failure and/or Yahoo have disconnected your connection owing to excessive bandwidth use...

To check to see if you have been "toasted", try visiting your "my groups" page... this will show "Error 999" if your IP has been toasted!

The only way to restore the download is to disconnect and reconnect to the internet (or try using "--user" and "--pass" and the same command, unless you have a "fixed IP" this should get another bite at the cherry if you list archive is public access, if you have a "Fixed IP" I don't know how long your IP is blocked (Tell me if you know)

You will be wise to use "--resume" if you ever use "--start" just in case you get toasted or your line goes down, if you run the same command it will start from where it fell over NOT from zero!!!

Top

back to the top


Go on...vote for Fatblokeracing.org .... ORIGINAL CONTENT © John "Fatbloke" Bullas | To Link to FBR click HERE | HTMLBLUNDERS? Click Here!!!! please