talk@lists.collectionspace.org

WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org

View all threads

Installation / data import

LH
Linda Hocking
Fri, Mar 9, 2012 2:10 PM

We would like to install Collectionspace but are trying to figure out
how to import data from the collection database it's currently in. We
have been using io, created by Willoughby now owned by Selago. The only
information they will give us is that we can export each table to excel.
It's an access based product. We are working with a contract programmer
who will install and set up collectionspace, but he is not familiar with
SPECTRUM, and we are hoping someone can help us figure out how to use
the data we already have. What we are wondering is if we should have the
programmers install a blank database, and then work on exporting the
tables and mapping the data, or if we need to do something to prepare
the data first. Many thanks for any suggestions!

Linda M. Hocking

Curator of Library & Archives

Litchfield Historical Society

P.O. Box 385

Litchfield, CT 06759

860-567-4501

http://www.litchfieldhistoricalsociety.org

archivist@litchfieldhistoricalsociety.org

We would like to install Collectionspace but are trying to figure out how to import data from the collection database it's currently in. We have been using io, created by Willoughby now owned by Selago. The only information they will give us is that we can export each table to excel. It's an access based product. We are working with a contract programmer who will install and set up collectionspace, but he is not familiar with SPECTRUM, and we are hoping someone can help us figure out how to use the data we already have. What we are wondering is if we should have the programmers install a blank database, and then work on exporting the tables and mapping the data, or if we need to do something to prepare the data first. Many thanks for any suggestions! Linda M. Hocking Curator of Library & Archives Litchfield Historical Society P.O. Box 385 Litchfield, CT 06759 860-567-4501 http://www.litchfieldhistoricalsociety.org archivist@litchfieldhistoricalsociety.org
AS
Angela Spinazze
Fri, Mar 9, 2012 4:04 PM

Hi Linda,

Thanks for your questions.

Here are a few resources that will help your process and a couple of
suggestions of how you might get started. I am sure that others will
chime in as well with suggestions.

  1. I would recommend that you start with a data mapping from your
    existing fields to the fields in CollectionSpace.  There is a sample
    mapping template that we've prepared and it is available here:
    http://wiki.collectionspace.org/display/deploy/New+Implementer+Jump+Start

You might also look at the sample implementation plan for guidance.

If you'd like to see examples of other museum mappings, have a look at
these two pages:
http://wiki.collectionspace.org/display/deploy/SMK
and
http://wiki.collectionspace.org/display/deploy/MMI+Data+Analysis+and+Mapping

  1. Yes, to installing an empty instance of CollectionSpace.
    It will be helpful to do this for a number of reasons including so
    that you and your team of end users will have a chance to check your
    mapping against what you see on the screen. I find it is very helpful
    to see where data are being mapped to rather than just a list of
    fields on a page.

  2. I wouldn't be too concerned about knowing about SPECTRUM.  We use
    it as a starting point for our data schema.  The most important
    documentation for the software engineer installing CollectionSpace is
    available here:
    http://wiki.collectionspace.org/display/DOC/CollectionSpace+Release+Documentation

This documentation relates to the current public release v2.0.
With each new release, new documentation is added, so, you'll notice
that there are pages for 2.1 and we're currently working on releasing
2.3. I would urge you to start with v2.0

  1. You may want to look around at the deployments section of the wiki
    to see what others are doing and how they are developing their plans
    and documentation.  There is a lot of good material there.

  2. There is a page about using Talend Open Studio for data migration
    that might also be helpful.
    http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Talend+Open+Studio+-+DRAFT

I hope that helps!  Keep the questions coming!

Angela

On Mar 9, 2012, at 8:10 AM, Linda Hocking wrote:

We would like to install Collectionspace but are trying to figure
out how to import data from the collection database it's currently
in. We have been using io, created by Willoughby now owned by
Selago. The only information they will give us is that we can export
each table to excel. It's an access based product. We are working
with a contract programmer who will install and set up
collectionspace, but he is not familiar with SPECTRUM, and we are
hoping someone can help us figure out how to use the data we already
have. What we are wondering is if we should have the programmers
install a blank database, and then work on exporting the tables and
mapping the data, or if we need to do something to prepare the data
first. Many thanks for any suggestions!

Linda M. Hocking
Curator of Library & Archives
Litchfield Historical Society
P.O. Box 385
Litchfield, CT 06759
860-567-4501
http://www.litchfieldhistoricalsociety.org
archivist@litchfieldhistoricalsociety.org


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

Hi Linda, Thanks for your questions. Here are a few resources that will help your process and a couple of suggestions of how you might get started. I am sure that others will chime in as well with suggestions. 1. I would recommend that you start with a data mapping from your existing fields to the fields in CollectionSpace. There is a sample mapping template that we've prepared and it is available here: http://wiki.collectionspace.org/display/deploy/New+Implementer+Jump+Start You might also look at the sample implementation plan for guidance. If you'd like to see examples of other museum mappings, have a look at these two pages: http://wiki.collectionspace.org/display/deploy/SMK and http://wiki.collectionspace.org/display/deploy/MMI+Data+Analysis+and+Mapping 2. Yes, to installing an empty instance of CollectionSpace. It will be helpful to do this for a number of reasons including so that you and your team of end users will have a chance to check your mapping against what you see on the screen. I find it is very helpful to see where data are being mapped to rather than just a list of fields on a page. 3. I wouldn't be too concerned about knowing about SPECTRUM. We use it as a starting point for our data schema. The most important documentation for the software engineer installing CollectionSpace is available here: http://wiki.collectionspace.org/display/DOC/CollectionSpace+Release+Documentation This documentation relates to the current public release v2.0. With each new release, new documentation is added, so, you'll notice that there are pages for 2.1 and we're currently working on releasing 2.3. I would urge you to start with v2.0 4. You may want to look around at the deployments section of the wiki to see what others are doing and how they are developing their plans and documentation. There is a lot of good material there. 5. There is a page about using Talend Open Studio for data migration that might also be helpful. http://wiki.collectionspace.org/display/deploy/Data+Migration+using+Talend+Open+Studio+-+DRAFT I hope that helps! Keep the questions coming! Angela On Mar 9, 2012, at 8:10 AM, Linda Hocking wrote: > We would like to install Collectionspace but are trying to figure > out how to import data from the collection database it's currently > in. We have been using io, created by Willoughby now owned by > Selago. The only information they will give us is that we can export > each table to excel. It's an access based product. We are working > with a contract programmer who will install and set up > collectionspace, but he is not familiar with SPECTRUM, and we are > hoping someone can help us figure out how to use the data we already > have. What we are wondering is if we should have the programmers > install a blank database, and then work on exporting the tables and > mapping the data, or if we need to do something to prepare the data > first. Many thanks for any suggestions! > > > Linda M. Hocking > Curator of Library & Archives > Litchfield Historical Society > P.O. Box 385 > Litchfield, CT 06759 > 860-567-4501 > http://www.litchfieldhistoricalsociety.org > archivist@litchfieldhistoricalsociety.org > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
CH
Chris Hoffman
Fri, Mar 9, 2012 4:46 PM

Hi Linda,

I'd only add a couple points to Angela's excellent suggestions, and I'll ask a question or two as well.

It is really important to figure out what parts of CollectionSpace you want to use and which ones need data migrated into them.  Will you be importing in information about loan transactions for example, or deaccessions?

When you install CollectionSpace, you basically get an empty system. It might come with some values in drop downs already initialized, but otherwise, you've got an empty database to start with.  At UC Berkeley, we've found that we usually start by importing some of the key authorities -- persons, organizations, storage locations (if those are all important to you).  That's important because when you import other data, you need more than the person's name in order to put a person in a certain field (such as the authorizer of a loan transaction).  You need a form of the name called the refname.  On Cataloging, you can get away with very little of this refname business.  It all depends on which fields on the Cataloging page need information from your existing system.

So here are my questions: How many objects do you have in your system now, and which procedures (transactions) will also require data migration (loans in, loans out, acquisitions, intake, object exit, media)?

Thanks,
Chris

Chris Hoffman, Ph.D.
Manager of Informatics Services
IST-Research & Content Technologies, UC Berkeley
chris.hoffman@berkeley.edu
510-642-9643

On Mar 9, 2012, at 6:10 AM, Linda Hocking wrote:

We would like to install Collectionspace but are trying to figure out how to import data from the collection database it's currently in. We have been using io, created by Willoughby now owned by Selago. The only information they will give us is that we can export each table to excel. It's an access based product. We are working with a contract programmer who will install and set up collectionspace, but he is not familiar with SPECTRUM, and we are hoping someone can help us figure out how to use the data we already have. What we are wondering is if we should have the programmers install a blank database, and then work on exporting the tables and mapping the data, or if we need to do something to prepare the data first. Many thanks for any suggestions!

Linda M. Hocking
Curator of Library & Archives
Litchfield Historical Society
P.O. Box 385
Litchfield, CT 06759
860-567-4501
http://www.litchfieldhistoricalsociety.org
archivist@litchfieldhistoricalsociety.org


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

Hi Linda, I'd only add a couple points to Angela's excellent suggestions, and I'll ask a question or two as well. It is really important to figure out what parts of CollectionSpace you want to use and which ones need data migrated into them. Will you be importing in information about loan transactions for example, or deaccessions? When you install CollectionSpace, you basically get an empty system. It might come with some values in drop downs already initialized, but otherwise, you've got an empty database to start with. At UC Berkeley, we've found that we usually start by importing some of the key authorities -- persons, organizations, storage locations (if those are all important to you). That's important because when you import other data, you need more than the person's name in order to put a person in a certain field (such as the authorizer of a loan transaction). You need a form of the name called the refname. On Cataloging, you can get away with very little of this refname business. It all depends on which fields on the Cataloging page need information from your existing system. So here are my questions: How many objects do you have in your system now, and which procedures (transactions) will also require data migration (loans in, loans out, acquisitions, intake, object exit, media)? Thanks, Chris Chris Hoffman, Ph.D. Manager of Informatics Services IST-Research & Content Technologies, UC Berkeley chris.hoffman@berkeley.edu 510-642-9643 On Mar 9, 2012, at 6:10 AM, Linda Hocking wrote: > We would like to install Collectionspace but are trying to figure out how to import data from the collection database it's currently in. We have been using io, created by Willoughby now owned by Selago. The only information they will give us is that we can export each table to excel. It's an access based product. We are working with a contract programmer who will install and set up collectionspace, but he is not familiar with SPECTRUM, and we are hoping someone can help us figure out how to use the data we already have. What we are wondering is if we should have the programmers install a blank database, and then work on exporting the tables and mapping the data, or if we need to do something to prepare the data first. Many thanks for any suggestions! > > > Linda M. Hocking > Curator of Library & Archives > Litchfield Historical Society > P.O. Box 385 > Litchfield, CT 06759 > 860-567-4501 > http://www.litchfieldhistoricalsociety.org > archivist@litchfieldhistoricalsociety.org > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
SS
Susan Stone
Fri, Mar 9, 2012 5:51 PM

Linda,

I don't know if you already know this or if someone else has explained
it, but I believe the way most collections migrate data from exiting
systems is by using the cspace import service to load data into cspace.
In order to do that they create XML documents from their existing data
following cspace XML schemas. ETL tools like Talend and Pentaho Kettle
can help you do this for data in a database or data in text files (like
csv files created from excel files), and maybe directly from Excel files.

Susan

On 03/09/2012 06:10 AM, Linda Hocking wrote:

We would like to install Collectionspace but are trying to figure out
how to import data from the collection database it's currently in. We
have been using io, created by Willoughby now owned by Selago. The only
information they will give us is that we can export each table to excel.
It's an access based product. We are working with a contract programmer
who will install and set up collectionspace, but he is not familiar with
SPECTRUM, and we are hoping someone can help us figure out how to use
the data we already have. What we are wondering is if we should have the
programmers install a blank database, and then work on exporting the
tables and mapping the data, or if we need to do something to prepare
the data first. Many thanks for any suggestions!

Linda M. Hocking

Curator of Library & Archives

Litchfield Historical Society

P.O. Box 385

Litchfield, CT 06759

860-567-4501

http://www.litchfieldhistoricalsociety.org

archivist@litchfieldhistoricalsociety.org


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

Linda, I don't know if you already know this or if someone else has explained it, but I believe the way most collections migrate data from exiting systems is by using the cspace import service to load data into cspace. In order to do that they create XML documents from their existing data following cspace XML schemas. ETL tools like Talend and Pentaho Kettle can help you do this for data in a database or data in text files (like csv files created from excel files), and maybe directly from Excel files. Susan On 03/09/2012 06:10 AM, Linda Hocking wrote: > We would like to install Collectionspace but are trying to figure out > how to import data from the collection database it's currently in. We > have been using io, created by Willoughby now owned by Selago. The only > information they will give us is that we can export each table to excel. > It's an access based product. We are working with a contract programmer > who will install and set up collectionspace, but he is not familiar with > SPECTRUM, and we are hoping someone can help us figure out how to use > the data we already have. What we are wondering is if we should have the > programmers install a blank database, and then work on exporting the > tables and mapping the data, or if we need to do something to prepare > the data first. Many thanks for any suggestions! > > Linda M. Hocking > > Curator of Library & Archives > > Litchfield Historical Society > > P.O. Box 385 > > Litchfield, CT 06759 > > 860-567-4501 > > http://www.litchfieldhistoricalsociety.org > > archivist@litchfieldhistoricalsociety.org > > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
LH
Linda Hocking
Fri, Mar 9, 2012 8:06 PM

Many thanks to Angela, Chris and Susan for responding to my questions.
To answer some of yours, there are 6620 records in our existing
database. There is a loans module where some data regarding loans
exists. It doesn't generate the paperwork but records that an item was
out. There is an exhibits module that tracks what items were on display
and for what dates. There are three authority areas- people,
publications, and thesaurus. The location seems to be part of the item
record and not an authority file. We're hoping to use features
CollectionSpace provides that we have either not had in our current
software (or underused because they are unwieldy.) I think the most
important thing we need to get out of one and into the other are the
records for the objects and the associated images. The export does
create csv files, so we will need to figure out how to convert that to
XML. We have one other quirk- the archives and the object collections
basically come from the same people and they document the same people.
Our authorities duplicate each other. Is there a way to merge the two,
or point the name authorities from collectionSpace to the files stored
in Archon? Again, many thanks for your help!

Linda

On 03/09/2012 06:10 AM, Linda Hocking wrote:

We would like to install Collectionspace but are trying to figure out
how to import data from the collection database it's currently in. We
have been using io, created by Willoughby now owned by Selago. The

only

information they will give us is that we can export each table to

excel.

It's an access based product. We are working with a contract

programmer

who will install and set up collectionspace, but he is not familiar

with

SPECTRUM, and we are hoping someone can help us figure out how to use
the data we already have. What we are wondering is if we should have

the

programmers install a blank database, and then work on exporting the
tables and mapping the data, or if we need to do something to prepare
the data first. Many thanks for any suggestions!

Linda M. Hocking

Curator of Library & Archives

Litchfield Historical Society

P.O. Box 385

Litchfield, CT 06759

860-567-4501

http://www.litchfieldhistoricalsociety.org

archivist@litchfieldhistoricalsociety.org


Talk mailing list
Talk@lists.collectionspace.org

Many thanks to Angela, Chris and Susan for responding to my questions. To answer some of yours, there are 6620 records in our existing database. There is a loans module where some data regarding loans exists. It doesn't generate the paperwork but records that an item was out. There is an exhibits module that tracks what items were on display and for what dates. There are three authority areas- people, publications, and thesaurus. The location seems to be part of the item record and not an authority file. We're hoping to use features CollectionSpace provides that we have either not had in our current software (or underused because they are unwieldy.) I think the most important thing we need to get out of one and into the other are the records for the objects and the associated images. The export does create csv files, so we will need to figure out how to convert that to XML. We have one other quirk- the archives and the object collections basically come from the same people and they document the same people. Our authorities duplicate each other. Is there a way to merge the two, or point the name authorities from collectionSpace to the files stored in Archon? Again, many thanks for your help! Linda On 03/09/2012 06:10 AM, Linda Hocking wrote: > We would like to install Collectionspace but are trying to figure out > how to import data from the collection database it's currently in. We > have been using io, created by Willoughby now owned by Selago. The only > information they will give us is that we can export each table to excel. > It's an access based product. We are working with a contract programmer > who will install and set up collectionspace, but he is not familiar with > SPECTRUM, and we are hoping someone can help us figure out how to use > the data we already have. What we are wondering is if we should have the > programmers install a blank database, and then work on exporting the > tables and mapping the data, or if we need to do something to prepare > the data first. Many thanks for any suggestions! > > Linda M. Hocking > > Curator of Library & Archives > > Litchfield Historical Society > > P.O. Box 385 > > Litchfield, CT 06759 > > 860-567-4501 > > http://www.litchfieldhistoricalsociety.org > > archivist@litchfieldhistoricalsociety.org > > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections pace.org