Hi all,

Thanks to the instructions at

ld+or+group+of+fields (and the related pages) I've just been able to

new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to

inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.

My main outstanding issue now is that I get some unexpected results

adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the

as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they

be correctly configured?

Cheers,
Chris

Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)

Talk mailing list
Talk@lists.collectionspace.org

Hi Chris,

I've had no responses other than the ones on the list. I've yet to
follow up on the hint to check out the Persons schema.

For data loading, I'm currently using the CS Services API from

Talend Open Studio. I'm using the JAXB jars from CS to help with the
field mapping and building the requests using the Jersey library
(although it would make more sense to use the CS Java Client library

this). The decision to use Services API was partly so I could get to
know the API and partly because I prefer the feedback I get - if the
load fails, it's much easier to find out what went wrong. Nuxeo

import would sometimes fail and I had no idea how far I had got or

record had caused the problem.

We have about 33,000 artworks and these have been imported. If I
remember correctly this takes 4-5 hours. I'm afraid I haven't timed

imports accurately, I'm just trying to remember roughly how long it
takes. I've also imported persons (artists and featured persons),
movement records, intake records, acquistion records, vocabularies

relations between these. Each of these took 'hours' spread over a

of weeks. I've not attempted the whole lot in a single job.

I had supposed that the collectionobject records would represent the
greatest load because they contain the most data but it appears that

overhead for an operation on a record (ie. waiting for a http

is taking the most time and that the amount of data in a record is

so significant. So it's all the relations that are taking the most

(especially movement relations as there can be many movements for

work and I create a relation in each direction). I may end with
generating records and using the CS 'import' functionality if this
proves faster.

All the above was on CS version 1.0b. We've recently switched to 1.3

currently I'm working on schema extension rather than importing so I
don't have many records loaded. Just ask if you've any more

if anything is not clear...

Regards,
Chris

-----Oprindelig meddelelse-----
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 2. februar 2011 19:11
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject

Hi Chris,

It's great to hear about your success with schema extensions!  We've
been doing a lot of work with the UCJEPS herbaria on extending the
collectionobject schema to add fields useful for herbaria and other
natural history collections.

Did anyone reply to your question about boolean fields?

We're experimenting with loading some larger data volumes into a

1.0 instance, and I'm wondering what you are doing with data

How are you doing data import at this point?  Are you still creating

records and loading via the nuxeo shell?  How many records have you
loaded into your instance?  And what version of CSpace are you

pace.org

Hi Chris,

I've had no responses other than the ones on the list. I've yet to
follow up on the hint to check out the Persons schema.

For data loading, I'm currently using the CS Services API from

Talend Open Studio. I'm using the JAXB jars from CS to help with the
field mapping and building the requests using the Jersey library
(although it would make more sense to use the CS Java Client library

this). The decision to use Services API was partly so I could get to
know the API and partly because I prefer the feedback I get - if the
load fails, it's much easier to find out what went wrong. Nuxeo

import would sometimes fail and I had no idea how far I had got or

record had caused the problem.

We have about 33,000 artworks and these have been imported. If I
remember correctly this takes 4-5 hours. I'm afraid I haven't timed

imports accurately, I'm just trying to remember roughly how long it
takes. I've also imported persons (artists and featured persons),
movement records, intake records, acquistion records, vocabularies

relations between these. Each of these took 'hours' spread over a

of weeks. I've not attempted the whole lot in a single job.

I had supposed that the collectionobject records would represent the
greatest load because they contain the most data but it appears that

overhead for an operation on a record (ie. waiting for a http

is taking the most time and that the amount of data in a record is

so significant. So it's all the relations that are taking the most

(especially movement relations as there can be many movements for

work and I create a relation in each direction). I may end with
generating records and using the CS 'import' functionality if this
proves faster.

All the above was on CS version 1.0b. We've recently switched to 1.3

currently I'm working on schema extension rather than importing so I
don't have many records loaded. Just ask if you've any more

if anything is not clear...

Regards,
Chris

-----Oprindelig meddelelse-----
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 2. februar 2011 19:11
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject

Hi Chris,

It's great to hear about your success with schema extensions!  We've
been doing a lot of work with the UCJEPS herbaria on extending the
collectionobject schema to add fields useful for herbaria and other
natural history collections.

Did anyone reply to your question about boolean fields?

We're experimenting with loading some larger data volumes into a

1.0 instance, and I'm wondering what you are doing with data

How are you doing data import at this point?  Are you still creating

records and loading via the nuxeo shell?  How many records have you
loaded into your instance?  And what version of CSpace are you

pace.org

Talk mailing list
Talk@lists.collectionspace.org

(I think we should change the name of this mailing list to

collectionspace-list!)

-----Oprindelig meddelelse-----
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 3. februar 2011 19:03
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Loading data

Hi Chris (Pott),

(I think we should change the name of this mailing list to

collectionspace-list!)

Thanks so much for this good information about data loading.  It's

helpful, and we will look into the approach you described.  The
CollectionSpace roadmap shows some new work on import and export in

1.5 timeframe.

2.0
We will be keeping a close eye on this too.

Like you, we have also found that performance is heavily affected by

number of relations something has.  Another challenging

loading seems to be repeating fields.  If an object has multiple

names for instance, what is the best way to import that information?

the original create transaction or as an update?  We've talked a bit

the services team here at UCB, and they are thinking about using XML

represent these kinds of relationships, probably using

Talend or Kettle to create the XML information from data

database itself.  We haven't really explored this yet, but we will

soon.

Anyway, it's good to hear what you all are doing.  We'll talk again

Hi Chris,

I've attached a bunch of files containing the data you requested, I hope
they're pretty self explanatory. I noticed that....
*The chain GET seems to be correct
*The uischema shows the type of the new boolean field as "string"

Some extra info.....

In collectionobjects_smk.xsd;
<xs:element name="smkVerso" type="xs:boolean" />

In cspace-config.xml:
<field id="smkVerso" section="smk">
<selector>csc-smk-verso</selector>
</field>

In ObjectEntryTemplate.html:

Thanks for the help,
Chris

Fra: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sendt: 3. februar 2011 11:27
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject

Hi Chris,

Sorry for not replying to you earlier but I have had my head stuck in
config simplification.

My name is Chris and I work mainly with the App layer. Which is really
just a glue layer to allow the UI to talk in JSON and the Service layer
to talk in XML and everyone to understand each other. So from my point
of view it would be interesting to see what everyone is saying and
seeing if we are having some mis-communication between the layers.

Is there a way for me to see your UI interface? Because what would be
useful is to look at some of the payloads that are being passed between
the UI and the app. And to be able to mock up some calls to the service
layer so I can see what they are saying. If your UI doesn't have an
externally accessible URL could you do some triage for me and send me
the payloads
(this page is something I quickly put together, that I know needs help
and more info:
http://wiki.collectionspace.org/display/collectionspace/Guidelines+for+B
ug+Triage)
What I am most interested in would be the GET and POST/PUT calls
to /chain/{recordtype}/{csid} that get called when you load up an object
and save it.
If you could specify which is the boolean field and what you believe
ought to be the value in each payload that would be great. It would be
useful to have a payload where you would expect the boolean to be true
and also a second one where you expect it to be false.
And then a second call where the data would be helpful is
/cspace-services/{recordtype in the plural}/{csid}
which should return an xml payload that the services is sending and that
should help diagnose how the service layer is pushing the boolean value
back.

if you could also send me
/chain/{recordid}/uischema
and
/chain/{recordid}/uispec

I can see if the UI is initializing the fields correctly.

So if the value sent in the /chain/ POST/PUT is incorrect but /chain/
GET is correct then the most likely issue is with the UI but if the
schema and uispec aren't being set up correct then the issue might be in
the App not telling the UI to be boolean. However, /cspace-services/
call might not be passing the data initially in a format the App is
expecting.
Or it might just be that the underlying fluid framework in the UI hasn't
been set up to look for checkboxes.

This should be fun.

So any and all info would be really useful

Thanks for your time in this matter.
Chris

On 01/02/2011 09:30, Christopher Pott wrote:
Hi all,

Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.

My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?

Cheers,
Chris

Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)

Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org