Wednesday, December 28, 2005

Log: Complete replication in Magnolia

Main ToDo for today:
Till 10 PM.

1. Replication :
Each node periodically ( period depends on rate of failure I guess or rate of updates for documents/traffic ), copies a chunk of its documents, rehashes their id ( gets a 2nd/3rd etc hash id and just sends all the document information to the next node in the group....corresponding to that hash id.

The issue is how to compute the hash ids: do we do it individually for each document ? That would be quite expensive.

It is important to tie the replication to the documentId, not to the node, because the node that stores the document can change frequently.

Each node should have its independent replication period which changes according to number of updates/traffic and failure rate. Eventually...

* Implemented the replication function where the chunk size dynamically depends on the silbing table size....
* Now need to test it. start its events and see what happens, how does the replication go...

List of parameters ( more ) :
1. Query sibling size : How many siblings to send query to for each forwarded query....

2. Gift/Sift

----------------------LONG TERM------------------------
1. The fingertables generation currently is centralized. Not acceptable !! Need a join function with finger table entry update.
For the paper it may not matter, but is important for the final implementation....
Comments: Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?