Need help with mirroring MxO Official Websites

Okay, so I have unlimited hosting, unlimited domains, and unlimited bandwith. I can register "www.DataNodeOne-Mirror.com" or something like that.

If someone can tell me how to mirror a site, I'll be happy to do it for the sake of posterity.

I can't think of any automated way to mirror a site, but there probably is a script or program somewhere that can trawl a site, copy its contents and generate the necessary new links. That wouldn't work with something like a forum, though, 'cos it's always changing, and you'd end up with a pretty mess that you'd have to clean up.

And then there are the copyright concerns. ¬_¬

I thought mirroring a site was legal. Jeez, now I have to research this . . .

Maybe there's some brain out there who's smart enough to figure out the right command... Looks like I am still missing something. Possibly I should add the --allowed-sites parameter. Well. I'm too lazy to figure it out but I guess that's at least something for you to start with.

wget --mirror -p --html-extension --convert-links -r --level=3 -k -P/usr/mxo --random-wait --user-agent="" -e robots=off http://forums.station.sony.com/mxo/forums/list.m

Silenci0 wrote:

Maybe there's some brain out there who's smart enough to figure out the right command... Looks like I am still missing something. Possibly I should add the --allowed-sites parameter. Well. I'm too lazy to figure it out but I guess that's at least something for you to start with.

wget --mirror -p --html-extension --convert-links -r --level=3 -k -P/usr/mxo --random-wait --user-agent="" -e robots=off http://forums.station.sony.com/mxo/forums/list.m

Okay, now we're talking. But can you be a little more step-by-step for us novices lol?

Ntrails wrote:

It would be cool if all the old posts could be archived somewhere for when I get bored.

Seriously, It is therapeutic for me.

   Agreed.  And I think certain forums (like the Z.I.C., the Zero One Network, the EPN Relay, etc.) should also be archived for their relation to the overall series of events in the story (such as LESIG posts and the like).

should be done for sure, I like the idea wish I could help somehow

Give this a try:  http://www.httrack.com/

Bingo! Got it, once again, SJ comes through with a blinding brilliance of intellect.

Now, since this little gadget is fairly straight-forward, any ideas about names?

Bummer, the flash page is already gone.

so this might be posable to do then?   what do you mean by "names"? names for what..

www.datanodeone-archived.com or www.datanodeonemirror.com and such. . .

Well, I got it all, including the flash page.

The forums, however, are a different story. Apparently it's not as easy to mirror forums.

ShiXinFeng wrote:

Well, I got it all, including the flash page.

The forums, however, are a different story. Apparently it's not as easy to mirror forums.

I am the admin of ZIC and can give you full access and admin rights to it for a cookie

So, considering what's going on with the servers right now, this might be the last day (smiley?)...


has anyone achieved anything with the forum mirroring project?

If they just disappeared tomorrow, without any further notice... without anything saved...   well, I'd survive it, but it would suck.


I remember someone archived the old version of these forums (I think they were Lithium forums), and they were actually complete and interactive.

So any progress with that? Just curious.

I haven't been able to mirror more than the forums/list.m page. It's a bit frustrating since there is no support for this kind of thing from the HTtrack software people.

HTT Track allows you to grab the site, then update it with the changes, so you are not trying to get it all at the last moment.. 

Its all in the settings..

the issue you are having is you are following the robots.txt its in the settings to ignore robots.txt.

what about legal issues? mirroring the website should be fine as long as u do not publish it when soe decides to shut it down. am i rait? :o