Hi, As promised, I've started a page on the wiki to try and explain one of the methods used for performing migration at SMK. Feel free to edit and comment, I'd realy like to start a discussion about the best way to approach this so as much code/mapping as possible can be shared. The page can be found here: http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Tale nd+Open+Studio+-+DRAFT Regards, Chris Developer, Corpus Project Statens Museum for Kunst (National Gallery of Denmark)

Hi Chris, Wow! This is a really helpful document, and I can tell it took significant work on your part. Many thanks. We'll study this right away. I know that Susan Stone is interested in testing whether Kettle (also called Spoon, also called Pentaho Data Integration) can perform these same tasks. She's used it to extract and transform data for the Phoebe A. Hearst Museum of Anthropology. Have you looked at Kettle at all? Thanks, Chris On Feb 8, 2011, at 3:00 AM, Christopher Pott wrote: > Hi, > > As promised, I've started a page on the wiki to try and explain one of > the methods used for performing migration at SMK. Feel free to edit and > comment, I'd realy like to start a discussion about the best way to > approach this so as much code/mapping as possible can be shared. The > page can be found here: > > http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Tale > nd+Open+Studio+-+DRAFT > > Regards, > Chris > > Developer, Corpus Project > Statens Museum for Kunst (National Gallery of Denmark) > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

Hi Chris, Yes, I downloaded and tried Kettle whilst I was evaluating Talend. I never really made a decision not use Kettle but ended up using Talend because: *it could connect to one of our data sources which Kettle couldn't *it's straightforward to import/reuse Java code from Collectionspace *it's based on Eclipse which I was (a little) familiar with *as a product it seemed more 'polished' I suspect Talend has a steeper learning curve but this could be shortened if we built some commmon modules or libraries which could be shared. But I look forward to hear about experiences with Kettle (if anyone wants to add some examples to the wiki it would be helpful to compare). Cheers, Chris > -----Oprindelig meddelelse----- > Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu] > Sendt: 8. februar 2011 18:07 > Til: Christopher Pott > Cc: talk@lists.collectionspace.org > Emne: Re: [Talk] Data migration wiki page > > Hi Chris, > > Wow! This is a really helpful document, and I can tell it took > significant work on your part. Many thanks. We'll study this right away. > I know that Susan Stone is interested in testing whether Kettle (also > called Spoon, also called Pentaho Data Integration) can perform these same > tasks. She's used it to extract and transform data for the Phoebe A. > Hearst Museum of Anthropology. Have you looked at Kettle at all? > > Thanks, > Chris > > On Feb 8, 2011, at 3:00 AM, Christopher Pott wrote: > > > Hi, > > > > As promised, I've started a page on the wiki to try and explain one of > > the methods used for performing migration at SMK. Feel free to edit and > > comment, I'd realy like to start a discussion about the best way to > > approach this so as much code/mapping as possible can be shared. The > > page can be found here: > > > > http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Tale > > nd+Open+Studio+-+DRAFT > > > > Regards, > > Chris > > > > Developer, Corpus Project > > Statens Museum for Kunst (National Gallery of Denmark) > > > > > > _______________________________________________ > > Talk mailing list > > Talk@lists.collectionspace.org > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections pa > ce.org

If we go with Chris's java approach, then I think Talend is probably the better choice. Kettle is very intuitive for playing with data though. Susan On 02/08/2011 01:07 PM, Christopher Pott wrote: > Hi Chris, > > Yes, I downloaded and tried Kettle whilst I was evaluating Talend. I > never really made a decision not use Kettle but ended up using Talend > because: > > *it could connect to one of our data sources which Kettle couldn't > *it's straightforward to import/reuse Java code from Collectionspace > *it's based on Eclipse which I was (a little) familiar with > *as a product it seemed more 'polished' > > I suspect Talend has a steeper learning curve but this could be > shortened if we built some commmon modules or libraries which could be > shared. But I look forward to hear about experiences with Kettle (if > anyone wants to add some examples to the wiki it would be helpful to > compare). > > Cheers, > Chris > >> -----Oprindelig meddelelse----- >> Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu] >> Sendt: 8. februar 2011 18:07 >> Til: Christopher Pott >> Cc: talk@lists.collectionspace.org >> Emne: Re: [Talk] Data migration wiki page >> >> Hi Chris, >> >> Wow! This is a really helpful document, and I can tell it took >> significant work on your part. Many thanks. We'll study this right > away. >> I know that Susan Stone is interested in testing whether Kettle (also >> called Spoon, also called Pentaho Data Integration) can perform these > same >> tasks. She's used it to extract and transform data for the Phoebe A. >> Hearst Museum of Anthropology. Have you looked at Kettle at all? >> >> Thanks, >> Chris >> >> On Feb 8, 2011, at 3:00 AM, Christopher Pott wrote: >> >>> Hi, >>> >>> As promised, I've started a page on the wiki to try and explain one > of >>> the methods used for performing migration at SMK. Feel free to edit > and >>> comment, I'd realy like to start a discussion about the best way to >>> approach this so as much code/mapping as possible can be shared. The >>> page can be found here: >>> >>> > http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Tale >>> nd+Open+Studio+-+DRAFT >>> >>> Regards, >>> Chris >>> >>> Developer, Corpus Project >>> Statens Museum for Kunst (National Gallery of Denmark) >>> >>> >>> _______________________________________________ >>> Talk mailing list >>> Talk@lists.collectionspace.org >>> >> > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections > pa >> ce.org > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org