http://cs.usfca.edu/~rstevens/archiveproject/ Updates about Chronica development. en-us 2004-12-03T09:47:13-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000532.html We have finalized the contents and status of Chronica for the final release.... rstevens 2004-12-03T09:47:13-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000529.html For those dependent on their mouse, we now have a GUI for setting the index parameters and beginning the indexing run.... rstevens 2004-11-28T17:33:19-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000521.html After some annoying debug runs, I've gotten a portable version of the wayback machine going, for locally stored arc files. Yay. For now I'm borrowing archive's javascript, which, while covering most stuff, misses a lot of other off-site references...I may... defendio 2004-11-05T23:48:54-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000505.html Here is the API documentation for Chronica so far, some comments need to be updated, etc. Will be updated.... rstevens 2004-10-20T16:51:08-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000502.html Time-plot search The time-plot search is working. I set the images up using this gpl chart package I found on sourceforge, and it'll graph any query or combination of queries that you give it (see the "need help?" button... defendio 2004-10-19T03:57:29-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000501.html The current searchable index contains 2,194,824 documents. Thats the combination of the UK docs from the IA, as well as the local crawls done here at USF. The total indexing took 57 hours to do 1,473 arc files totaling 104GB... cfrasche 2004-10-18T18:05:26-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000500.html I added the XML config reader to the respository and added support to ARCRecordReader...... rstevens 2004-10-16T00:45:03-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000499.html Since we're now able to index real data that's already in the wayback machine, I set it up to link straight to it...and it works. As long as the page is in the wayback machine, the links will work. Feel... defendio 2004-10-15T12:09:08-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000498.html Well, i can't find the cd with the back up of my inward links code... crap. Anyway, it's not as bad as it sounds because i was probably going to rewrite the whole thing anyway. The code had a lot... jendo 2004-10-12T00:29:39-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000492.html From an email i sent earlier to day: Fellas, http://chronica.cs.usfca.edu/chronica/src/ that address will point you to the src that the web ui is currently running off of... it's essentially our 'stable' build. There's some jsp pages in another directory that... cfrasche 2004-10-06T14:43:32-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000490.html I ran one of our larger arc files with 215 pdf and 1287 html formatted docs... I timed each parse and calculated the average parse time for each .. PDF: 3.89 seconds HTML: 0.019 seconds... cfrasche 2004-10-02T18:47:26-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000487.html So I finally had enough iterations through indexing the large set of arc files we've gathere and cleaning up bugs and errors in the code/files to where I was comfortable letting the index finish and to try searching on it.... cfrasche 2004-10-01T01:42:37-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000486.html Release 0.1 of Chronica is now active....... rstevens 2004-09-30T14:35:43-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000485.html After taking the time to clean up the summaries and titles of the html parser otuput, I moved on to adding more functionality.. PDFBox was the solution to pdf files. So the current index built off a small crawl from... cfrasche 2004-09-30T01:50:23-08:00 http://cs.usfca.edu/~rstevens/archiveproject/archives/000480.html Yeah, so check your email, I fixed the weird character bug in the ARR.extractText().... My bad, collecting the read data wrong.... I think we are good to go for thursday....... rstevens 2004-09-28T00:25:20-08:00