Revision as of 13:45, 8 November 2005

Re-Implementing the MMS for GEM

Phase I

Current implementation has very tight ties to an Oracle database. For new, generalized MMS, the functions need to be more abstract, in order to work with any database. The internal user management systems were handed off to an external service in the original implementation. In the re-implementation, roles and rights for users need to be abstracted to work either wholly within the MMS or linked to an external user authentication system.

1. New implementation of the main Repository database
   • Currently based on Oracle 9i. We need to decide whether to go with Oracle's new release (10g called 'Express Edition') which is free for limited use, or switch to MySQL. Regardless of which is chosen for the first re-implementation, the code will be developed to make that decision a local one, with the MMS working with any database.
   • One advantage of sticking with Oracle is that they are coming out with a native RDF database. Another advantage is that some Oracle specific development done for the NSDL can be re-used.
2. New implementation of the internal management database
   • The current database has been developed over time and is due for re-factoring, though the core ideas are the same. It is currently based on Qualified DC and it's not clear that anything else would be necessary for this portion of the internal work.
3. Abstracting user and service management and adding to internal management database (using registry model)
   • User management was based on an incomplete integration with SourceForge which made it impossible to manage users effectively. Service management was limited for some of the same reasons, although internally we set up a framework that was managed manually.
   • A new user management module needs to be developed that can be easily integrated into other user management systems. This will allow the building of a service management based on lessons learned in the NSDL implementation.
4. Setup of simple services structure (similar to Recommender)
   • Current recommender too closely integrated into the MMS management database
   • New implementation will be a separate service harvested into the Repository
   • Other services should be set up on the same model: harvest, change, expose. This includes human mediated services such as editing and alignment services. CWIS might be an effective start on this (though should be pre-configured).
5. Implement new harvester
   • New harvester should be based on existing harvester, modified to be more forgiving of invalid XML, that will report errors down to specific line, enabling effective notification. Acceptance should cascade from full harvest to statement-by-statement, only rejecting the smallest possible portions of a potential harvest.

Phase II

1. Re-implementing the MMS OAI server (exising server needs to have data backend made more abstract to deal with any data schema or source)
2. Adjust the metadata editor to accommodate any metadata schema and work with the simple services structure
3. Implement services registry on top of existing components (user management and repository)
4. Implement error handling and notifications
5. Setup safe transformation service, generalized to any repository

Phase III

1. Implement harvest diagnostics and helpdesk functionality
2. Implement distributed GEM community services (includes OAI server capable of working on many platforms, plugin data connections, etc.)
3. Implement comprehensive evaluation and test suite, validity checkers

Phase IV

1. Implement metadata augmentation, improvement and rating services
2. Non-OAI data services
3. Secure OAI services