Difference between revisions of "Re-implementing the MMS"

From Metadata-Registry
Jump to: navigation, search
Line 17: Line 17:
 
#* New implementation will be a separate service harvested into the Repository
 
#* New implementation will be a separate service harvested into the Repository
 
#* Other services should be set up on the same model: harvest, change, expose. This includes human mediated services such as editing and alignment services.  CWIS might be an effective start on this (though should be pre-configured).
 
#* Other services should be set up on the same model: harvest, change, expose. This includes human mediated services such as editing and alignment services.  CWIS might be an effective start on this (though should be pre-configured).
#Implement new harvester (JP: estimate 3 weeks)
+
#Implement new harvester (JP estimate: 3 weeks)
 
#* New harvester should be based on existing harvester, modified to be more forgiving of invalid XML, that will report errors down to specific line, enabling effective notification.  Acceptance should cascade from full harvest to statement-by-statement, only rejecting the smallest possible portions of a potential harvest.
 
#* New harvester should be based on existing harvester, modified to be more forgiving of invalid XML, that will report errors down to specific line, enabling effective notification.  Acceptance should cascade from full harvest to statement-by-statement, only rejecting the smallest possible portions of a potential harvest.
  

Revision as of 09:52, 12 December 2005

Re-Implementing the MMS for GEM

Phase I

Current implementation has very tight ties to an Oracle database. For new, generalized MMS, the functions need to be more abstract, in order to work with any database. The internal user management systems were handed off to an external service in the original implementation. In the re-implementation, roles and rights for users need to be abstracted to work either wholly within the MMS or linked to an external user authentication system.

  1. New implementation of the main Repository database (JP estimate: 3 days)
    • Currently based on Oracle 9i. We need to decide whether to go with Oracle's new release (10g called 'Express Edition') which is free for limited use, or switch to MySQL. Regardless of which is chosen for the first re-implementation, the code will be developed to make that decision a local one, with the MMS working with any database.
    • One advantage of sticking with Oracle is that they are coming out with a native RDF database. Another advantage is that some Oracle specific development done for the NSDL can be re-used.
  2. New implementation of the internal management database (JP estimate: 3 days)
    • The current database has been developed over time and is due for re-factoring, though the core ideas are the same. It is currently based on Qualified DC and it's not clear that anything else would be necessary for this portion of the internal work.
  3. Abstracting user and service management and adding to internal management database (using registry model) (JP estimate: 4 weeks)
    • User management was based on an incomplete integration with SourceForge which made it impossible to manage users effectively. Service management was limited for some of the same reasons, although internally we set up a framework that was managed manually.
    • A new user management module needs to be developed that can be easily integrated into other user management systems. This will allow the building of a service management based on lessons learned in the NSDL implementation.
  4. Setup of simple services structure (similar to Recommender) (JP estimate: 2 weeks)
    • Current recommender too closely integrated into the MMS management database
    • New implementation will be a separate service harvested into the Repository
    • Other services should be set up on the same model: harvest, change, expose. This includes human mediated services such as editing and alignment services. CWIS might be an effective start on this (though should be pre-configured).
  5. Implement new harvester (JP estimate: 3 weeks)
    • New harvester should be based on existing harvester, modified to be more forgiving of invalid XML, that will report errors down to specific line, enabling effective notification. Acceptance should cascade from full harvest to statement-by-statement, only rejecting the smallest possible portions of a potential harvest.

Phase II

  1. Re-implementing the MMS OAI server
    • Exising server needs to have data backend made more abstract to deal with any data schema or source
  2. Adjust the metadata editor to accommodate any metadata schema and work with the simple services structure
    • Current metadata editor tightly tied to qualified Dublin Core (flat and non-hierarchical) and with the database. Needs to stand alone as a separate service.
    • Enabling editing for hierarchical schemas (such as LOM) will require additional work
  3. Implement services registry on top of existing components (user management and repository)
    • Needs to make a distinction between services and service providers, which doesn't currently exist.
    • Current service setup is unidirectional (only works with services it controls). Also needs to register service interfaces that can be managed by the MMS (Lenny)
  4. Implement error handling and notifications
    • Agents choose preferred notification type and methodology (email, RSS, etc.)
    • Defaults are configured by agent role
  5. Setup safe transformation service, generalized to any repository
    • Another simple service separately configured
    • Based on NSDL safe transforms

Phase III

  1. Implement comprehensive harvest diagnostics, validity checkers and helpdesk functionality
    • New or changed harvest generate statistical summary and sample or full export file for evaluation tool (generated on the fly)
    • May use separate third party software for helpdesk
  2. Implement distributed GEM community services (includes OAI server capable of working on many platforms, plugin data connections, etc.)
  3. Implement collection-specific transforms

Phase IV

  1. Implement metadata augmentation, improvement and rating services
  2. Non-OAI data services
  3. Secure OAI services