talk@lists.collectionspace.org
WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org
View all threadsadding new fields to collectionobject
Hi all,
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Hi Chris,
It's great to hear about your success with schema extensions! We've been doing a lot of work with the UCJEPS herbaria on extending the collectionobject schema to add fields useful for herbaria and other natural history collections.
Did anyone reply to your question about boolean fields?
We're experimenting with loading some larger data volumes into a CSpace 1.0 instance, and I'm wondering what you are doing with data loading. How are you doing data import at this point? Are you still creating XML records and loading via the nuxeo shell? How many records have you loaded into your instance? And what version of CSpace are you using?
Thanks,
Chris
On Feb 1, 2011, at 1:30 AM, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
On Tue, Feb 1, 2011 at 1:30 AM, Christopher Pott
Christopher.Pott@smk.dk wrote: wrote:
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. ... I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
Thanks for this report, Chris! We're delighted to hear that!
Beginning with release 1.4, the CollectionSpace project team has
begun working to further simplify configuration for extending schema
for domains/communities and individual museums. We are looking
forward to sharing this work with you and other implementers.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
and Chris Hoffman followed up by asking:
Did anyone reply to your question about boolean fields?
We have so far used only two underlying data types in the Nuxeo EP
repository, xs:string and xs:dateTime and have not yet experience with
using xs:boolean (or its underlying representation in MySQL).
So far, boolean / binary options have been represented by simple,
two-term vocabularies in application configuration, and by pull-down
menus presenting the options in the UI, although checkboxes may also
have their place. It'd be interesting to know more about your use
case and examples.
Aron
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean.
I originally responded:
We have so far used only two underlying data types in the Nuxeo EP
repository, xs:string and xs:dateTime and have not yet experience with
using xs:boolean (or its underlying representation in MySQL).
My statement above is flatly incorrect. As Patrick has pointed out,
we already use a Boolean value in authorities schemae, such as this
excerpt from the persons_common.xsd file:
<xs:element name="displayNameComputed" type="xs:boolean"/>
Round-tripping these values between the services layer and the
repository (Nuxeo/MySQL) has been working successfully.
Aron
Hi Chris,
I've had no responses other than the ones on the list. I've yet to
follow up on the hint to check out the Persons schema.
For data loading, I'm currently using the CS Services API from within
Talend Open Studio. I'm using the JAXB jars from CS to help with the
field mapping and building the requests using the Jersey library
(although it would make more sense to use the CS Java Client library for
this). The decision to use Services API was partly so I could get to
know the API and partly because I prefer the feedback I get - if the
load fails, it's much easier to find out what went wrong. Nuxeo shell
import would sometimes fail and I had no idea how far I had got or which
record had caused the problem.
We have about 33,000 artworks and these have been imported. If I
remember correctly this takes 4-5 hours. I'm afraid I haven't timed the
imports accurately, I'm just trying to remember roughly how long it
takes. I've also imported persons (artists and featured persons),
movement records, intake records, acquistion records, vocabularies and
relations between these. Each of these took 'hours' spread over a number
of weeks. I've not attempted the whole lot in a single job.
I had supposed that the collectionobject records would represent the
greatest load because they contain the most data but it appears that the
overhead for an operation on a record (ie. waiting for a http response)
is taking the most time and that the amount of data in a record is not
so significant. So it's all the relations that are taking the most time
(especially movement relations as there can be many movements for each
work and I create a relation in each direction). I may end with
generating records and using the CS 'import' functionality if this
proves faster.
All the above was on CS version 1.0b. We've recently switched to 1.3 but
currently I'm working on schema extension rather than importing so I
don't have many records loaded. Just ask if you've any more questions or
if anything is not clear...
Regards,
Chris
-----Oprindelig meddelelse-----
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 2. februar 2011 19:11
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject
Hi Chris,
It's great to hear about your success with schema extensions! We've
been doing a lot of work with the UCJEPS herbaria on extending the
collectionobject schema to add fields useful for herbaria and other
natural history collections.
Did anyone reply to your question about boolean fields?
We're experimenting with loading some larger data volumes into a CSpace
1.0 instance, and I'm wondering what you are doing with data loading.
How are you doing data import at this point? Are you still creating XML
records and loading via the nuxeo shell? How many records have you
loaded into your instance? And what version of CSpace are you using?
Thanks,
Chris
On Feb 1, 2011, at 1:30 AM, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
ld+or+group+of+fields (and the related pages) I've just been able to
add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to
my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results
when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the
UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they
should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
Hi Chris,
Sorry for not replying to you earlier but I have had my head stuck in
config simplification.
My name is Chris and I work mainly with the App layer. Which is really
just a glue layer to allow the UI to talk in JSON and the Service layer
to talk in XML and everyone to understand each other. So from my point
of view it would be interesting to see what everyone is saying and
seeing if we are having some mis-communication between the layers.
Is there a way for me to see your UI interface? Because what would be
useful is to look at some of the payloads that are being passed between
the UI and the app. And to be able to mock up some calls to the service
layer so I can see what they are saying. If your UI doesn't have an
externally accessible URL could you do some triage for me and send me
the payloads
(this page is something I quickly put together, that I know needs help
and more info:
http://wiki.collectionspace.org/display/collectionspace/Guidelines+for+Bug+Triage)
What I am most interested in would be the GET and POST/PUT calls
to /chain/{recordtype}/{csid} that get called when you load up an object
and save it.
If you could specify which is the boolean field and what you believe
ought to be the value in each payload that would be great. It would be
useful to have a payload where you would expect the boolean to be true
and also a second one where you expect it to be false.
And then a second call where the data would be helpful is
/cspace-services/{recordtype in the plural}/{csid}
which should return an xml payload that the services is sending and that
should help diagnose how the service layer is pushing the boolean value
back.
if you could also send me
/chain/{recordid}/uischema
and
/chain/{recordid}/uispec
I can see if the UI is initializing the fields correctly.
So if the value sent in the /chain/ POST/PUT is incorrect but /chain/
GET is correct then the most likely issue is with the UI but if the
schema and uispec aren't being set up correct then the issue might be in
the App not telling the UI to be boolean. However, /cspace-services/
call might not be passing the data initially in a format the App is
expecting.
Or it might just be that the underlying fluid framework in the UI hasn't
been set up to look for checkboxes.
This should be fun.
So any and all info would be really useful
Thanks for your time in this matter.
Chris
On 01/02/2011 09:30, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Hi Chris,
I've attached a bunch of files containing the data you requested, I hope
they're pretty self explanatory. I noticed that....
*The chain GET seems to be correct
*The uischema shows the type of the new boolean field as "string"
Some extra info.....
In collectionobjects_smk.xsd;
<xs:element name="smkVerso" type="xs:boolean" />
In cspace-config.xml:
<field id="smkVerso" section="smk">
<selector>csc-smk-verso</selector>
</field>
In ObjectEntryTemplate.html:
<div class="info-pair"> <div class="header"> <div class="label">Verso</div> </div> <div class="content" //also tried "checkbox" here <input type="checkbox" class="csc-smk-verso"/> </div> </div>Thanks for the help,
Chris
Fra: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sendt: 3. februar 2011 11:27
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject
Hi Chris,
Sorry for not replying to you earlier but I have had my head stuck in
config simplification.
My name is Chris and I work mainly with the App layer. Which is really
just a glue layer to allow the UI to talk in JSON and the Service layer
to talk in XML and everyone to understand each other. So from my point
of view it would be interesting to see what everyone is saying and
seeing if we are having some mis-communication between the layers.
Is there a way for me to see your UI interface? Because what would be
useful is to look at some of the payloads that are being passed between
the UI and the app. And to be able to mock up some calls to the service
layer so I can see what they are saying. If your UI doesn't have an
externally accessible URL could you do some triage for me and send me
the payloads
(this page is something I quickly put together, that I know needs help
and more info:
http://wiki.collectionspace.org/display/collectionspace/Guidelines+for+B
ug+Triage)
What I am most interested in would be the GET and POST/PUT calls
to /chain/{recordtype}/{csid} that get called when you load up an object
and save it.
If you could specify which is the boolean field and what you believe
ought to be the value in each payload that would be great. It would be
useful to have a payload where you would expect the boolean to be true
and also a second one where you expect it to be false.
And then a second call where the data would be helpful is
/cspace-services/{recordtype in the plural}/{csid}
which should return an xml payload that the services is sending and that
should help diagnose how the service layer is pushing the boolean value
back.
if you could also send me
/chain/{recordid}/uischema
and
/chain/{recordid}/uispec
I can see if the UI is initializing the fields correctly.
So if the value sent in the /chain/ POST/PUT is incorrect but /chain/
GET is correct then the most likely issue is with the UI but if the
schema and uispec aren't being set up correct then the issue might be in
the App not telling the UI to be boolean. However, /cspace-services/
call might not be passing the data initially in a format the App is
expecting.
Or it might just be that the underlying fluid framework in the UI hasn't
been set up to look for checkboxes.
This should be fun.
So any and all info would be really useful
Thanks for your time in this matter.
Chris
On 01/02/2011 09:30, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org
ok so I think I see what is happening.
the app sends the UI :
"smkVerso": "false",
and the UI sends the app: "smkVerso":["true"] or "smkVerso":[] for a
false value.
This suggests that the UI is expecting an array, I need to confirm with
yura whether an array is the expected behaviour for a boolean value. And
whether the uischema needs to be tweaked so it specifies boolean instead
of string.
If an array is not the expect format for the UI then I suspect yura will
have to look into what needs to be changed from an UI point of view but
if that is the required format then I can quickly extend the App to
correctly deal with booleans and return an array and then convert that
back into a string for the services.
It seems to be that it is checking the checkbox if anything other than
an empty array is returned. Am I right in assuming that when you create
a collectionobject that the checkbox is not checked?
Am I right in assuming you are working with v1.3 build or is it an
earlier build?
Chris
On 03/02/2011 12:29, Christopher Pott wrote:
Hi Chris,
I've attached a bunch of files containing the data you requested, I hope
they're pretty self explanatory. I noticed that....
*The chain GET seems to be correct
*The uischema shows the type of the new boolean field as "string"
Some extra info.....
In collectionobjects_smk.xsd;
<xs:element name="smkVerso" type="xs:boolean" />
In cspace-config.xml:
<field id="smkVerso" section="smk">
<selector>csc-smk-verso</selector>
</field>
In ObjectEntryTemplate.html:
<div class="info-pair"> <div class="header"> <div class="label">Verso</div> </div> <div class="content" //also tried "checkbox" here <input type="checkbox" class="csc-smk-verso"/> </div> </div>Thanks for the help,
Chris
Fra: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sendt: 3. februar 2011 11:27
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject
Hi Chris,
Sorry for not replying to you earlier but I have had my head stuck in
config simplification.
My name is Chris and I work mainly with the App layer. Which is really
just a glue layer to allow the UI to talk in JSON and the Service layer
to talk in XML and everyone to understand each other. So from my point
of view it would be interesting to see what everyone is saying and
seeing if we are having some mis-communication between the layers.
Is there a way for me to see your UI interface? Because what would be
useful is to look at some of the payloads that are being passed between
the UI and the app. And to be able to mock up some calls to the service
layer so I can see what they are saying. If your UI doesn't have an
externally accessible URL could you do some triage for me and send me
the payloads
(this page is something I quickly put together, that I know needs help
and more info:
http://wiki.collectionspace.org/display/collectionspace/Guidelines+for+B
ug+Triage)
What I am most interested in would be the GET and POST/PUT calls
to /chain/{recordtype}/{csid} that get called when you load up an object
and save it.
If you could specify which is the boolean field and what you believe
ought to be the value in each payload that would be great. It would be
useful to have a payload where you would expect the boolean to be true
and also a second one where you expect it to be false.
And then a second call where the data would be helpful is
/cspace-services/{recordtype in the plural}/{csid}
which should return an xml payload that the services is sending and that
should help diagnose how the service layer is pushing the boolean value
back.
if you could also send me
/chain/{recordid}/uischema
and
/chain/{recordid}/uispec
I can see if the UI is initializing the fields correctly.
So if the value sent in the /chain/ POST/PUT is incorrect but /chain/
GET is correct then the most likely issue is with the UI but if the
schema and uispec aren't being set up correct then the issue might be in
the App not telling the UI to be boolean. However, /cspace-services/
call might not be passing the data initially in a format the App is
expecting.
Or it might just be that the underlying fluid framework in the UI hasn't
been set up to look for checkboxes.
This should be fun.
So any and all info would be really useful
Thanks for your time in this matter.
Chris
On 01/02/2011 09:30, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
http://wiki.collectionspace.org/display/collectionspace/How+to+add+a+fie
ld+or+group+of+fields (and the related pages) I've just been able to add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org
Hi,
Yes, it's version v1.3.
If I "Create New" then the checkbox is empty. If I then "Save" it
becomes checked.
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sendt: 3. februar 2011 14:46
Til: Christopher Pott
Cc: talk@lists.collectionspace.org; Zenevich, Yura
Emne: Re: SV: [Talk] adding new fields to collectionobject
ok so I think I see what is happening.
the app sends the UI :
"smkVerso": "false",
and the UI sends the app: "smkVerso":["true"] or "smkVerso":[] for a
false value.
This suggests that the UI is expecting an array, I need to confirm with
yura whether an array is the expected behaviour for a boolean value. And
whether the uischema needs to be tweaked so it specifies boolean instead
of string.
If an array is not the expect format for the UI then I suspect yura will
have to look into what needs to be changed from an UI point of view but
if that is the required format then I can quickly extend the App to
correctly deal with booleans and return an array and then convert that
back into a string for the services.
It seems to be that it is checking the checkbox if anything other than
an empty array is returned. Am I right in assuming that when you create
a collectionobject that the checkbox is not checked?
Am I right in assuming you are working with v1.3 build or is it an
earlier build?
Chris
On 03/02/2011 12:29, Christopher Pott wrote:
Hi Chris,
I've attached a bunch of files containing the data you requested, I
hope
they're pretty self explanatory. I noticed that....
*The chain GET seems to be correct
*The uischema shows the type of the new boolean field as "string"
Some extra info.....
In collectionobjects_smk.xsd;
<xs:element name="smkVerso" type="xs:boolean" />
In cspace-config.xml:
<field id="smkVerso" section="smk">
<selector>csc-smk-verso</selector>
</field>
In ObjectEntryTemplate.html:
<div class="info-pair"> <div class="header"> <div class="label">Verso</div> </div> <div class="content" //also tried "checkbox" here <input type="checkbox" class="csc-smk-verso"/> </div> </div>Thanks for the help,
Chris
Fra: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sendt: 3. februar 2011 11:27
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject
Hi Chris,
Sorry for not replying to you earlier but I have had my head stuck in
config simplification.
My name is Chris and I work mainly with the App layer. Which is really
just a glue layer to allow the UI to talk in JSON and the Service
layer
to talk in XML and everyone to understand each other. So from my point
of view it would be interesting to see what everyone is saying and
seeing if we are having some mis-communication between the layers.
Is there a way for me to see your UI interface? Because what would be
useful is to look at some of the payloads that are being passed
between
the UI and the app. And to be able to mock up some calls to the
service
layer so I can see what they are saying. If your UI doesn't have an
externally accessible URL could you do some triage for me and send me
the payloads
(this page is something I quickly put together, that I know needs help
and more info:
ug+Triage)
What I am most interested in would be the GET and POST/PUT calls
to /chain/{recordtype}/{csid} that get called when you load up an
object
and save it.
If you could specify which is the boolean field and what you believe
ought to be the value in each payload that would be great. It would be
useful to have a payload where you would expect the boolean to be true
and also a second one where you expect it to be false.
And then a second call where the data would be helpful is
/cspace-services/{recordtype in the plural}/{csid}
which should return an xml payload that the services is sending and
that
should help diagnose how the service layer is pushing the boolean
value
back.
if you could also send me
/chain/{recordid}/uischema
and
/chain/{recordid}/uispec
I can see if the UI is initializing the fields correctly.
So if the value sent in the /chain/ POST/PUT is incorrect but /chain/
GET is correct then the most likely issue is with the UI but if the
schema and uispec aren't being set up correct then the issue might be
in
the App not telling the UI to be boolean. However, /cspace-services/
call might not be passing the data initially in a format the App is
expecting.
Or it might just be that the underlying fluid framework in the UI
hasn't
been set up to look for checkboxes.
This should be fun.
So any and all info would be really useful
Thanks for your time in this matter.
Chris
On 01/02/2011 09:30, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
ld+or+group+of+fields (and the related pages) I've just been able to
add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to
my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results
when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the
UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they
should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org
pace.org
Hi Chris (Pott),
(I think we should change the name of this mailing list to the chris-collectionspace-list!)
Thanks so much for this good information about data loading. It's very helpful, and we will look into the approach you described. The CollectionSpace roadmap shows some new work on import and export in the 1.5 timeframe.
https://wiki.collectionspace.org:8444/display/collectionspace/Road+Map+to+2.0
We will be keeping a close eye on this too.
Like you, we have also found that performance is heavily affected by the number of relations something has. Another challenging area for data loading seems to be repeating fields. If an object has multiple object names for instance, what is the best way to import that information? With the original create transaction or as an update? We've talked a bit with the services team here at UCB, and they are thinking about using XML to represent these kinds of relationships, probably using something like Talend or Kettle to create the XML information from data files or from the database itself. We haven't really explored this yet, but we will have to soon.
Anyway, it's good to hear what you all are doing. We'll talk again soon on the chris-collectionspace-list :-)
Chris (Hoffman)
On Feb 3, 2011, at 2:05 AM, Christopher Pott wrote:
Hi Chris,
I've had no responses other than the ones on the list. I've yet to
follow up on the hint to check out the Persons schema.
For data loading, I'm currently using the CS Services API from within
Talend Open Studio. I'm using the JAXB jars from CS to help with the
field mapping and building the requests using the Jersey library
(although it would make more sense to use the CS Java Client library for
this). The decision to use Services API was partly so I could get to
know the API and partly because I prefer the feedback I get - if the
load fails, it's much easier to find out what went wrong. Nuxeo shell
import would sometimes fail and I had no idea how far I had got or which
record had caused the problem.
We have about 33,000 artworks and these have been imported. If I
remember correctly this takes 4-5 hours. I'm afraid I haven't timed the
imports accurately, I'm just trying to remember roughly how long it
takes. I've also imported persons (artists and featured persons),
movement records, intake records, acquistion records, vocabularies and
relations between these. Each of these took 'hours' spread over a number
of weeks. I've not attempted the whole lot in a single job.
I had supposed that the collectionobject records would represent the
greatest load because they contain the most data but it appears that the
overhead for an operation on a record (ie. waiting for a http response)
is taking the most time and that the amount of data in a record is not
so significant. So it's all the relations that are taking the most time
(especially movement relations as there can be many movements for each
work and I create a relation in each direction). I may end with
generating records and using the CS 'import' functionality if this
proves faster.
All the above was on CS version 1.0b. We've recently switched to 1.3 but
currently I'm working on schema extension rather than importing so I
don't have many records loaded. Just ask if you've any more questions or
if anything is not clear...
Regards,
Chris
-----Oprindelig meddelelse-----
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 2. februar 2011 19:11
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] adding new fields to collectionobject
Hi Chris,
It's great to hear about your success with schema extensions! We've
been doing a lot of work with the UCJEPS herbaria on extending the
collectionobject schema to add fields useful for herbaria and other
natural history collections.
Did anyone reply to your question about boolean fields?
We're experimenting with loading some larger data volumes into a CSpace
1.0 instance, and I'm wondering what you are doing with data loading.
How are you doing data import at this point? Are you still creating XML
records and loading via the nuxeo shell? How many records have you
loaded into your instance? And what version of CSpace are you using?
Thanks,
Chris
On Feb 1, 2011, at 1:30 AM, Christopher Pott wrote:
Hi all,
Thanks to the instructions at
ld+or+group+of+fields (and the related pages) I've just been able to
add
new local and domain (finearts) schemas to extend collectionobject. It
took me longer than it probably should have but this was only due to
my
inability to slow down and read the instructions properly. Although I
knew the intention was to make configuration straightforward, I was
still surprised to see how simple the process was (ie. no coding
required) and was really impressed to see the new (text) fields up and
working so quickly and the new content being saved and searched.
My main outstanding issue now is that I get some unexpected results
when
adding new fields of type boolean. These fields are defined in the xsd
as xs:boolean, represented in mySql as bit(1) and implemented in the
UI
as checkboxes. My problem is that whatever the state of the checkbox
when saved, "false" is always returned by chain and the box is always
checked upon reload. As there are no boolean fields in the current
schemas could someone please help me with an example of how they
should
be correctly configured?
Cheers,
Chris
Developer, Corpus Project
Statens Museum for Kunst (National Gallery of Denmark)
Talk mailing list
Talk@lists.collectionspace.org