| Index
» Forums
» Free download sitemaps. html site maps |
 |
Platon.SK discussion forums |
|
| View previous topic :: View next topic |
| Author |
Message |
Asdszeer Guest
|
Posted: 2007-12-17 00:25 Post subject: Free download sitemaps. html site maps |
|
|
html site map
You may run into problems downloading files of unusual size. Some older operating systems, file systems, and web clients have a hard limit of 2GB on file size. If you seem to be hitting this limit, try using wget version 1.10 or greater, cURL version 7.11.1-1 or greater, or a recent version of lynx (using -dump). Users have experienced problems with Mozilla and Firefox, but recent versions are more likely to be fixed.
It is recommended that you check the MD5 sums (provided in a file in the download directory, to make sure your download was complete and accurate. You can check this by running the "md5sum" command on the files you downloaded. Given how large the files are, this may take some time to calculate. Due to the technical details of how files are stored, file sizes may be reported differently on different filesystems, and so are not necessarily reliable. Also, you may have experienced corruption during the download, though this is unlikely.
html site map
Suppose you are building a piece of software that at certain points displays information that came from wikipedia. If you want your program to display the information in a different way than can be seen in the live version, you'll probably need the wikicode that is used to enter it, instead of the finished HTML.
Also if you want to get all of the data, you'll probably want to transfer it in the most efficient way that's possible. The wikipedia.org servers need to do quite a bit of work to convert the wikicode into html. That's time consuming both for you and for the wikipedia.org servers, so simply spidering all pages is not the way to go.
html site map
html site map |
|
| Back to top |
|
 |
|
|
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|