AR
Aron Roberts
Tue, Oct 13, 2015 9:15 PM
Hey Peter,
Ray and I just talked about this. This is what we think we figured out.
-
There's definitely confusion over the difference between
Personauthority and Person records.
-
Yes, Person records now have a repeatable term name group. As John just
noted :), the Gist that he helpfully created to paste in a Person import
example for PAHMA from "a few years ago" likely used an older schema, one
that pre-dated changes introduced c. mid-2012 in CollectionSpace version
2.4, per https://issues.collectionspace.org/browse/CSPACE-4996.
-
The selection of an example for creating Personauthority records, in
that Imports Service Home document, might be more confusing than helpful
... particularly as the example doesn't include any person records that
would then populate that authority with terms.
Generally, we're thinking one might now create Person authorities simply
by editing App layer config files and re-initializing authorities, although
as an alternative it's still possible to use the imports service for that
task. In any event, I'll make a documentation JIRA issue to fix up the
Imports Service Home document to reduce confusion, however we end up fixing
it.
- Ray's point about Person records needing to point to their parent
Personauthoriy, via setting the values of their persons_common:inAuthority
fields to the CSID of that parent Personauthority record, is an important
one: you'll definitely need to do that when creating Person records that
you want to import into a particular authority.
I hope I've got all this more or less right. Please feel free, John and
Ray, to jump in as you see fit ...
Aron
On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee rhlee@berkeley.edu wrote:
Peter,
The examples are indeed out of date. In older versions of cspace
displayName was a top-level field, but now it is in the repeating group.
To follow up on Aron's reply, the field that connects a person to a
personauthority is persons_common:inAuthority. You must set that to the
csid of the personauthority in which the person item should reside (e.g.
Local vs. ULAN). That could explain why your record isn't appearing.
Ray
On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts aron@socrates.berkeley.edu
wrote:
I'm not quite sure what the difference between "Persons" and
"Personauthorities" is, but there are indeed two different service bindings
in my tenant bindings file
True. Personauthorities are authorities for (aka controlled lists of)
person terms. Persons are individual terms ("items") in those lists. The
former can be conceived of as a container or bucket, for holding terms
related to a particular category of persons, and the latter as those person
terms that fill it.
If a specific person authority doesn't yet exist, you'd need to create it
before you could add terms to it. In the default configuration of
CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one
sample person authority is created, as specified in config files in the
Application layer, via the process of visiting the URL for initializing
authorities. Your own tenant's authorities might differ from those in the
sample tenants.
It might be helpful to start out with a different import, in part because
this one doesn't introduce any of the potential complexities regarding
authorities and authority items. Does the Object Exit example in that
Imports Service Home document work when you try it, for starters?
https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest
That is, if you copy that XML snippet to a different file and use 'curl'
with a command pretty much identical to the one you used for
personauthorities?
On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray pmurray@chillco.com
wrote:
Thank you, John -- I hope that will be helpful. One thing I noticed
right off the bat is the service
and type
attributes on the import
element:
<import service="Persons" type="Person" CSID="
75d9ceef-5ee5-4afe-93da-16cf34700427">
versus
<import seq="2" service="Personauthorities" type="Personauthority" CSID=
"11111111-2222-3333-4444-123456789012">
I'm not quite sure what the difference between "Persons" and
"Personauthorities" is, but there are indeed two different service bindings
in my tenant bindings file (reading from
https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. --
which itself seems outdated because I didn't find the file where the
documentation says the file would be).
Another difference is in the record structure itself. I think you might
be on an older version of CSpace because in my persons_common schema the
Display Name is in a repeated group and has a slightly different element
name:
<persons_common:displayName>G. D. Baron [Ute]</persons_common:
displayName>
versus
<persons_common:personTermGroupList><persons_common:personTermGroup> <
persons_common:termDisplayName>Perf Test Person Auth ${docID}</
persons_common:termDisplayName></persons_common:personTermGroup></
persons_common:personTermGroupList>
I found the latter by looking at what a record looks like in its raw
form and comparing it with the documentation. Unfortunately, these changes
don't seem to be enough to get the record imported in the right form.
Peter
On Oct 13, 2015, at 11:47 AM, John B. LOWE jblowe@berkeley.edu wrote:
Peter,
There are a number of different "styles" or "dialects" of XML that will
work with the IMPORT service, it was interesting to see yours.
I had a quick look and I noticed your IMPORT document
(authority-request.xml) seems to have two instances of the <
personauthorities_common:shortIdentifier> element and I'm not sure how
that can possibly work.
For comparison, I created a gist with 2 Person Authority records and 2
Relations records of the sort we used for the Hearst Museum ("PAHMA") a few
years ago. (For reasons I won't get into here, the examples do not use the
"sparse payload" approach -- they have lots of empty fields.)
https://gist.github.com/jblowe/2a99906195667c27a725
But hopefully they will give you the flavor of "our" dialect of IMPORT
xml...
Good hunting!
John
On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray pmurray@chillco.com
wrote:
I'm no further along, but I think I understand the pieces a little
better. I put the 'refName' element back in and changed the text of the
element to match what I think is our "Local Names" persons authority. I
haven't modified the instances of the person authority, so it is using the
one defined in base-authority-person.xml (
https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46).
My request document looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml
The response from CSpace looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml
...and I went looking for the document referenced in the CSpace
response; it looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml
When I look for the record in the UI (
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person),
it is not there. The only thing in the logs continues to be these warnings
in cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
The 'catalina.out' file also has these two lines, along with log lines
that state the request directory and the document directory are being
created in $CATALINA_HOME/temp
I have a feeling that there is still something missing in the import
request document, but I can't put my finger on it.
Peter
On Oct 12, 2015, at 5:52 PM, Aron Roberts aron@socrates.berkeley.edu
wrote:
At a brief and superficial glance, it seems possible the refName
values, in that example on the Imports Service Home page might be obsolete.
The RefName page (
https://wiki.collectionspace.org/display/DOC/RefName) gives this
example, for importing a person authority record:
urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local
Person Vocabulary'
Note that unlike the example on the Imports Service Home page, that
example includes a 'name' component. (As well, the URL in this example also
reflects the tenant domain name, which is current practice, IIRC.) Contrast
the (presumably correct) example above with this one, from the imports page:
urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf
Test Person Auth'
Perhaps try putting back the "<personauthorities_common:refName>"
element and using the RefName page's example (the first one above) -
tweaked as needed to reflect the Museum of Man's values - and see if the
import works?
This might or might not be relevant, but it's what first comes to
mind.
On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray pmurray@chillco.com
wrote:
I think I've gotten all of the CSpace configuration and UI changes
behind me. Just a few things left to clean up. I'm concentrating on data
loading now. I figured it would be best to get started with the example in
Imports Service Home
at the "Example Request" heading. I saved the XML for the persons
authority records to a file and ran the curl command
:
$ curl -X POST https://cspace.vagrant/cspace-services/imports
-i -u admin@museumofman.org:Administrator
-H "Content-Type: application/xml"
-T authority-request.xtml
Initially the error response I got back said "Could not process import
payload. Check XML markup for not-well-formed errors, elements not matching
import schema, etc." and this was logged on the cspace-services.log file:
ERROR [...]
[org.collectionspace.services.imports.TemplateExpander:492] ERROR calling
expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName
for Authority (too few tokens)
So I removed the "<personauthorities_common:refName>" element in the
XML file and tried again. The response I got back (with prettified XML)
was:
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Mon, 12 Oct 2015 20:05:02 GMT
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=15768000
Content-Type: application/xml
Content-Length: 998
Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9;
Path=/cspace-services/; HttpOnly
Vary: Accept-Encoding
<?xml version="1.0" encoding="UTF-8"?>
<import>
<msg>SUCCESS</msg>
<importedRecords>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
</importedRecord>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>11111111-2222-3333-4444-123456789012</csid>
</importedRecord>
</importedRecords>
<status>Success</status>
<totalRecordsImported>2</totalRecordsImported>
<numRecordsImportedByDocType>
<numRecordsImported>
<docType>Personauthority</docType>
<numRecords>2</numRecords>
</numRecordsImported>
</numRecordsImportedByDocType>
<report>READ:
/home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ:
/home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
</import>
...which looks like a successful import, BUT this is in
cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
...AND the document doesn't seem to be in the database. Based on the
csid
I would expect these URLs to work:
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person
...because the documents can't be found (also from
cspace-services.log):
ERROR [...]
[org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5.
Caught exception:Failed to get document
/sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
ERROR [...]
[org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
Could not find document with CSID=11111111-2222-3333-4444-123456789012.
Caught exception:Failed to get document
/sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
... and the records don't show up in a search. I feel like I'm
missing something here, but I can't quite put my finger on it. Suggestions?
Peter
Hey Peter,
Ray and I just talked about this. This is what we *think* we figured out.
- There's definitely confusion over the difference between
Personauthority and Person records.
- Yes, Person records now have a repeatable term name group. As John just
noted :), the Gist that he helpfully created to paste in a Person import
example for PAHMA from "a few years ago" likely used an older schema, one
that pre-dated changes introduced c. mid-2012 in CollectionSpace version
2.4, per https://issues.collectionspace.org/browse/CSPACE-4996.
- The selection of an example for creating Personauthority records, in
that Imports Service Home document, might be more confusing than helpful
... particularly as the example doesn't include any person records that
would then populate that authority with terms.
Generally, we're thinking one might now create Person authorities simply
by editing App layer config files and re-initializing authorities, although
as an alternative it's still possible to use the imports service for that
task. In any event, I'll make a documentation JIRA issue to fix up the
Imports Service Home document to reduce confusion, however we end up fixing
it.
- Ray's point about Person records needing to point to their parent
Personauthoriy, via setting the values of their persons_common:inAuthority
fields to the CSID of that parent Personauthority record, is an important
one: you'll definitely need to do that when creating Person records that
you want to import into a particular authority.
I hope I've got all this more or less right. Please feel free, John and
Ray, to jump in as you see fit ...
Aron
On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee <rhlee@berkeley.edu> wrote:
> Peter,
> The examples are indeed out of date. In older versions of cspace
> displayName was a top-level field, but now it is in the repeating group.
>
> To follow up on Aron's reply, the field that connects a person to a
> personauthority is persons_common:inAuthority. You must set that to the
> csid of the personauthority in which the person item should reside (e.g.
> Local vs. ULAN). That could explain why your record isn't appearing.
>
> Ray
>
>
>
> On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts <aron@socrates.berkeley.edu>
> wrote:
>
>> Peter writes:
>> > I'm not quite sure what the difference between "Persons" and
>> "Personauthorities" is, but there are indeed two different service bindings
>> in my tenant bindings file
>>
>> True. Personauthorities are authorities for (aka controlled lists of)
>> person terms. Persons are individual terms ("items") in those lists. The
>> former can be conceived of as a container or bucket, for holding terms
>> related to a particular category of persons, and the latter as those person
>> terms that fill it.
>>
>> If a specific person authority doesn't yet exist, you'd need to create it
>> before you could add terms to it. In the default configuration of
>> CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one
>> sample person authority is created, as specified in config files in the
>> Application layer, via the process of visiting the URL for initializing
>> authorities. Your own tenant's authorities might differ from those in the
>> sample tenants.
>>
>> It might be helpful to start out with a different import, in part because
>> this one doesn't introduce any of the potential complexities regarding
>> authorities and authority items. Does the Object Exit example in that
>> Imports Service Home document work when you try it, for starters?
>>
>>
>> https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest
>>
>> That is, if you copy that XML snippet to a different file and use 'curl'
>> with a command pretty much identical to the one you used for
>> personauthorities?
>>
>>
>>
>> On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray <pmurray@chillco.com>
>> wrote:
>>
>>> Thank you, John -- I hope that will be helpful. One thing I noticed
>>> right off the bat is the `service` and `type` attributes on the `import`
>>> element:
>>>
>>> <import service="Persons" type="Person" CSID="
>>> 75d9ceef-5ee5-4afe-93da-16cf34700427">
>>>
>>> versus
>>>
>>> <import seq="2" service="Personauthorities" type="Personauthority" CSID=
>>> "11111111-2222-3333-4444-123456789012">
>>> I'm not quite sure what the difference between "Persons" and
>>> "Personauthorities" is, but there are indeed two different service bindings
>>> in my tenant bindings file (reading from
>>> https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. --
>>> which itself seems outdated because I didn't find the file where the
>>> documentation says the file would be).
>>>
>>> Another difference is in the record structure itself. I think you might
>>> be on an older version of CSpace because in my persons_common schema the
>>> Display Name is in a repeated group and has a slightly different element
>>> name:
>>>
>>> <persons_common:displayName>G. D. Baron [Ute]</persons_common:
>>> displayName>
>>>
>>> versus
>>>
>>>
>>> <persons_common:personTermGroupList><persons_common:personTermGroup> <
>>> persons_common:termDisplayName>Perf Test Person Auth ${docID}</
>>> persons_common:termDisplayName></persons_common:personTermGroup></
>>> persons_common:personTermGroupList>
>>> I found the latter by looking at what a record looks like in its raw
>>> form and comparing it with the documentation. Unfortunately, these changes
>>> don't seem to be enough to get the record imported in the right form.
>>>
>>>
>>> Peter
>>>
>>>
>>> On Oct 13, 2015, at 11:47 AM, John B. LOWE <jblowe@berkeley.edu> wrote:
>>>
>>> Peter,
>>>
>>> There are a number of different "styles" or "dialects" of XML that will
>>> work with the IMPORT service, it was interesting to see yours.
>>>
>>> I had a quick look and I noticed your IMPORT document
>>> (authority-request.xml) seems to have two instances of the <
>>> personauthorities_common:shortIdentifier> element and I'm not sure how
>>> that can possibly work.
>>>
>>> For comparison, I created a gist with 2 Person Authority records and 2
>>> Relations records of the sort we used for the Hearst Museum ("PAHMA") a few
>>> years ago. (For reasons I won't get into here, the examples do not use the
>>> "sparse payload" approach -- they have lots of empty fields.)
>>>
>>> https://gist.github.com/jblowe/2a99906195667c27a725
>>>
>>> But hopefully they will give you the flavor of "our" dialect of IMPORT
>>> xml...
>>>
>>> Good hunting!
>>>
>>> John
>>>
>>> On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray <pmurray@chillco.com>
>>> wrote:
>>>
>>>> I'm no further along, but I think I understand the pieces a little
>>>> better. I put the 'refName' element back in and changed the text of the
>>>> element to match what I think is our "Local Names" persons authority. I
>>>> haven't modified the instances of the person authority, so it is using the
>>>> one defined in base-authority-person.xml (
>>>> https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46).
>>>> My request document looks like this:
>>>>
>>>>
>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml
>>>>
>>>> The response from CSpace looks like this:
>>>>
>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml
>>>>
>>>> ...and I went looking for the document referenced in the CSpace
>>>> response; it looks like this:
>>>>
>>>>
>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml
>>>>
>>>> When I look for the record in the UI (
>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person),
>>>> it is not there. The only thing in the logs continues to be these warnings
>>>> in cspace-services.log:
>>>>
>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>> ID for document, won't add
>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>> ID for document, won't add
>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
>>>>
>>>> The 'catalina.out' file also has these two lines, along with log lines
>>>> that state the request directory and the document directory are being
>>>> created in $CATALINA_HOME/temp
>>>>
>>>> I have a feeling that there is still something missing in the import
>>>> request document, but I can't put my finger on it.
>>>>
>>>>
>>>> Peter
>>>>
>>>> On Oct 12, 2015, at 5:52 PM, Aron Roberts <aron@socrates.berkeley.edu>
>>>> wrote:
>>>>
>>>> At a brief and superficial glance, it seems possible the refName
>>>> values, in that example on the Imports Service Home page might be obsolete.
>>>>
>>>> The RefName page (
>>>> https://wiki.collectionspace.org/display/DOC/RefName) gives this
>>>> example, for importing a person authority record:
>>>>
>>>> urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local
>>>> Person Vocabulary'
>>>>
>>>> Note that unlike the example on the Imports Service Home page, that
>>>> example includes a 'name' component. (As well, the URL in this example also
>>>> reflects the tenant domain name, which is current practice, IIRC.) Contrast
>>>> the (presumably correct) example above with this one, from the imports page:
>>>>
>>>> urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf
>>>> Test Person Auth'
>>>>
>>>> Perhaps try putting back the "<personauthorities_common:refName>"
>>>> element and using the RefName page's example (the first one above) -
>>>> tweaked as needed to reflect the Museum of Man's values - and see if the
>>>> import works?
>>>>
>>>> This might or might not be relevant, but it's what first comes to
>>>> mind.
>>>>
>>>> On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray <pmurray@chillco.com>
>>>> wrote:
>>>>
>>>>> I think I've gotten all of the CSpace configuration and UI changes
>>>>> behind me. Just a few things left to clean up. I'm concentrating on data
>>>>> loading now. I figured it would be best to get started with the example in
>>>>> [Imports Service Home](
>>>>> https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home)
>>>>> at the "Example Request" heading. I saved the XML for the persons
>>>>> authority records to a file and ran the `curl command`:
>>>>>
>>>>> $ curl -X POST https://cspace.vagrant/cspace-services/imports \
>>>>> -i -u admin@museumofman.org:Administrator \
>>>>> -H "Content-Type: application/xml" \
>>>>> -T authority-request.xtml
>>>>>
>>>>> Initially the error response I got back said "Could not process import
>>>>> payload. Check XML markup for not-well-formed errors, elements not matching
>>>>> import schema, etc." and this was logged on the cspace-services.log file:
>>>>>
>>>>> ERROR [...]
>>>>> [org.collectionspace.services.imports.TemplateExpander:492] ERROR calling
>>>>> expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName
>>>>> for Authority (too few tokens)
>>>>>
>>>>> So I removed the "<personauthorities_common:refName>" element in the
>>>>> XML file and tried again. The response I got back (with prettified XML)
>>>>> was:
>>>>>
>>>>> HTTP/1.1 100 Continue
>>>>>
>>>>> HTTP/1.1 200 OK
>>>>> Date: Mon, 12 Oct 2015 20:05:02 GMT
>>>>> Server: Apache-Coyote/1.1
>>>>> Strict-Transport-Security: max-age=15768000
>>>>> Content-Type: application/xml
>>>>> Content-Length: 998
>>>>> Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9;
>>>>> Path=/cspace-services/; HttpOnly
>>>>> Vary: Accept-Encoding
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <import>
>>>>> <msg>SUCCESS</msg>
>>>>> <importedRecords>
>>>>> <importedRecord>
>>>>> <doctype>Personauthority</doctype>
>>>>> <csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
>>>>> </importedRecord>
>>>>> <importedRecord>
>>>>> <doctype>Personauthority</doctype>
>>>>> <csid>11111111-2222-3333-4444-123456789012</csid>
>>>>> </importedRecord>
>>>>> </importedRecords>
>>>>> <status>Success</status>
>>>>> <totalRecordsImported>2</totalRecordsImported>
>>>>> <numRecordsImportedByDocType>
>>>>> <numRecordsImported>
>>>>> <docType>Personauthority</docType>
>>>>> <numRecords>2</numRecords>
>>>>> </numRecordsImported>
>>>>> </numRecordsImportedByDocType>
>>>>> <report>READ:
>>>>> /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ:
>>>>> /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
>>>>> </import>
>>>>>
>>>>>
>>>>> ...which looks like a successful import, BUT this is in
>>>>> cspace-services.log:
>>>>>
>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>>> ID for document, won't add
>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>>> ID for document, won't add
>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
>>>>>
>>>>>
>>>>> ...AND the document doesn't seem to be in the database. Based on the
>>>>> `csid` I would expect these URLs to work:
>>>>>
>>>>>
>>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person
>>>>>
>>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person
>>>>>
>>>>> ...because the documents can't be found (also from
>>>>> cspace-services.log):
>>>>>
>>>>> ERROR [...]
>>>>> [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
>>>>> Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5.
>>>>> Caught exception:Failed to get document
>>>>> /sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
>>>>> ERROR [...]
>>>>> [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
>>>>> Could not find document with CSID=11111111-2222-3333-4444-123456789012.
>>>>> Caught exception:Failed to get document
>>>>> /sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
>>>>>
>>>>>
>>>>> ... and the records don't show up in a search. I feel like I'm
>>>>> missing something here, but I can't quite put my finger on it. Suggestions?
>>>>>
>>>>>
>>>>> Peter
>>>>
>>>>
>>> --
>>> Peter Murray
>>> Dev/Ops Lead and Project Manager
>>> Cherry Hill Company
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>>
>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>
>>>
>>
>> _______________________________________________
>> Talk mailing list
>> Talk@lists.collectionspace.org
>>
>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>
>>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
>
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>
>
PM
Peter Murray
Tue, Oct 13, 2015 10:05 PM
Re-ordering your message, Aron, to start with the good news.
Yes! That does work -- I now have two Object_Exit records!
I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file
True. Personauthorities are authorities for (aka controlled lists of) person terms. Persons are individual terms ("items") in those lists. The former can be conceived of as a container or bucket, for holding terms related to a particular category of persons, and the latter as those person terms that fill it.
If a specific person authority doesn't yet exist, you'd need to create it before you could add terms to it. In the default configuration of CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one sample person authority is created, as specified in config files in the Application layer, via the process of visiting the URL for initializing authorities. Your own tenant's authorities might differ from those in the sample tenants.
Okay -- this helps. The "Local Persons" 'container' is defined in the App config, so I don't need to POST against the API with 'service="Personauthories"'. What my previous combinations seemed to be missing is to POST against the API with 'service="Persons"' plus including a <persons_common:inAuthority> tag with the CSID of the 'Local Persons' container. I'm trying that now.
Peter
Peter Murray
Dev/Ops Lead and Project Manager
Cherry Hill Company
Re-ordering your message, Aron, to start with the good news.
> On Oct 13, 2015, at 4:45 PM, Aron Roberts <aron@socrates.berkeley.edu> wrote:
>
> It might be helpful to start out with a different import, in part because this one doesn't introduce any of the potential complexities regarding authorities and authority items. Does the Object Exit example in that Imports Service Home document work when you try it, for starters?
>
> https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest <https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest>
>
> That is, if you copy that XML snippet to a different file and use 'curl' with a command pretty much identical to the one you used for personauthorities?
Yes! That does work -- I now have two Object_Exit records!
> Peter writes:
> > I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file
>
> True. Personauthorities are authorities for (aka controlled lists of) person terms. Persons are individual terms ("items") in those lists. The former can be conceived of as a container or bucket, for holding terms related to a particular category of persons, and the latter as those person terms that fill it.
>
> If a specific person authority doesn't yet exist, you'd need to create it before you could add terms to it. In the default configuration of CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one sample person authority is created, as specified in config files in the Application layer, via the process of visiting the URL for initializing authorities. Your own tenant's authorities might differ from those in the sample tenants.
Okay -- this helps. The "Local Persons" 'container' is defined in the App config, so I don't need to POST against the API with 'service="Personauthories"'. What my previous combinations seemed to be missing is to POST against the API with 'service="Persons"' _plus_ including a <persons_common:inAuthority> tag with the CSID of the 'Local Persons' container. I'm trying that now.
Peter
--
Peter Murray
Dev/Ops Lead and Project Manager
Cherry Hill Company
PM
Peter Murray
Tue, Oct 13, 2015 10:28 PM
On Oct 13, 2015, at 5:15 PM, Aron Roberts aron@socrates.berkeley.edu wrote:
Hey Peter,
Ray and I just talked about this. This is what we think we figured out.
-
There's definitely confusion over the difference between Personauthority and Person records.
-
Yes, Person records now have a repeatable term name group. As John just noted :), the Gist that he helpfully created to paste in a Person import example for PAHMA from "a few years ago" likely used an older schema, one that pre-dated changes introduced c. mid-2012 in CollectionSpace version 2.4, per https://issues.collectionspace.org/browse/CSPACE-4996 https://issues.collectionspace.org/browse/CSPACE-4996.
-
The selection of an example for creating Personauthority records, in that Imports Service Home document, might be more confusing than helpful ... particularly as the example doesn't include any person records that would then populate that authority with terms.
Generally, we're thinking one might now create Person authorities simply by editing App layer config files and re-initializing authorities, although as an alternative it's still possible to use the imports service for that task. In any event, I'll make a documentation JIRA issue to fix up the Imports Service Home document to reduce confusion, however we end up fixing it.
- Ray's point about Person records needing to point to their parent Personauthoriy, via setting the values of their persons_common:inAuthority fields to the CSID of that parent Personauthority record, is an important one: you'll definitely need to do that when creating Person records that you want to import into a particular authority.
I hope I've got all this more or less right. Please feel free, John and Ray, to jump in as you see fit ...
Aron
On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee <rhlee@berkeley.edu mailto:rhlee@berkeley.edu> wrote:
Peter,
The examples are indeed out of date. In older versions of cspace displayName was a top-level field, but now it is in the repeating group.
To follow up on Aron's reply, the field that connects a person to a personauthority is persons_common:inAuthority. You must set that to the csid of the personauthority in which the person item should reside (e.g. Local vs. ULAN). That could explain why your record isn't appearing.
Ray
On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts <aron@socrates.berkeley.edu mailto:aron@socrates.berkeley.edu> wrote:
Peter writes:
I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file
True. Personauthorities are authorities for (aka controlled lists of) person terms. Persons are individual terms ("items") in those lists. The former can be conceived of as a container or bucket, for holding terms related to a particular category of persons, and the latter as those person terms that fill it.
If a specific person authority doesn't yet exist, you'd need to create it before you could add terms to it. In the default configuration of CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one sample person authority is created, as specified in config files in the Application layer, via the process of visiting the URL for initializing authorities. Your own tenant's authorities might differ from those in the sample tenants.
It might be helpful to start out with a different import, in part because this one doesn't introduce any of the potential complexities regarding authorities and authority items. Does the Object Exit example in that Imports Service Home document work when you try it, for starters?
https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest
That is, if you copy that XML snippet to a different file and use 'curl' with a command pretty much identical to the one you used for personauthorities?
On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray <pmurray@chillco.com mailto:pmurray@chillco.com> wrote:
Thank you, John -- I hope that will be helpful. One thing I noticed right off the bat is the service
and type
attributes on the import
element:
<import service="Persons" type="Person" CSID="75d9ceef-5ee5-4afe-93da-16cf34700427">
versus
<import seq="2" service="Personauthorities" type="Personauthority" CSID="11111111-2222-3333-4444-123456789012">
I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file (reading from https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. -- which itself seems outdated because I didn't find the file where the documentation says the file would be).
Another difference is in the record structure itself. I think you might be on an older version of CSpace because in my persons_common schema the Display Name is in a repeated group and has a slightly different element name:
<persons_common:displayName>G. D. Baron [Ute]</persons_common:displayName>
versus
<persons_common:personTermGroupList>
<persons_common:personTermGroup>
<persons_common:termDisplayName>Perf Test Person Auth ${docID}</persons_common:termDisplayName>
</persons_common:personTermGroup>
</persons_common:personTermGroupList>
I found the latter by looking at what a record looks like in its raw form and comparing it with the documentation. Unfortunately, these changes don't seem to be enough to get the record imported in the right form.
Peter
On Oct 13, 2015, at 11:47 AM, John B. LOWE <jblowe@berkeley.edu mailto:jblowe@berkeley.edu> wrote:
Peter,
There are a number of different "styles" or "dialects" of XML that will work with the IMPORT service, it was interesting to see yours.
I had a quick look and I noticed your IMPORT document (authority-request.xml) seems to have two instances of the <personauthorities_common:shortIdentifier> element and I'm not sure how that can possibly work.
For comparison, I created a gist with 2 Person Authority records and 2 Relations records of the sort we used for the Hearst Museum ("PAHMA") a few years ago. (For reasons I won't get into here, the examples do not use the "sparse payload" approach -- they have lots of empty fields.)
https://gist.github.com/jblowe/2a99906195667c27a725 https://gist.github.com/jblowe/2a99906195667c27a725
But hopefully they will give you the flavor of "our" dialect of IMPORT xml...
Good hunting!
John
On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray <pmurray@chillco.com mailto:pmurray@chillco.com> wrote:
I'm no further along, but I think I understand the pieces a little better. I put the 'refName' element back in and changed the text of the element to match what I think is our "Local Names" persons authority. I haven't modified the instances of the person authority, so it is using the one defined in base-authority-person.xml (https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46 https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46). My request document looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml
The response from CSpace looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml
...and I went looking for the document referenced in the CSpace response; it looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml
When I look for the record in the UI (https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person), it is not there. The only thing in the logs continues to be these warnings in cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
The 'catalina.out' file also has these two lines, along with log lines that state the request directory and the document directory are being created in $CATALINA_HOME/temp
I have a feeling that there is still something missing in the import request document, but I can't put my finger on it.
Peter
On Oct 12, 2015, at 5:52 PM, Aron Roberts <aron@socrates.berkeley.edu mailto:aron@socrates.berkeley.edu> wrote:
At a brief and superficial glance, it seems possible the refName values, in that example on the Imports Service Home page might be obsolete.
The RefName page (https://wiki.collectionspace.org/display/DOC/RefName https://wiki.collectionspace.org/display/DOC/RefName) gives this example, for importing a person authority record:
urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local Person Vocabulary'
Note that unlike the example on the Imports Service Home page, that example includes a 'name' component. (As well, the URL in this example also reflects the tenant domain name, which is current practice, IIRC.) Contrast the (presumably correct) example above with this one, from the imports page:
urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf Test Person Auth'
Perhaps try putting back the "<personauthorities_common:refName>" element and using the RefName page's example (the first one above) - tweaked as needed to reflect the Museum of Man's values - and see if the import works?
This might or might not be relevant, but it's what first comes to mind.
On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray <pmurray@chillco.com mailto:pmurray@chillco.com> wrote:
I think I've gotten all of the CSpace configuration and UI changes behind me. Just a few things left to clean up. I'm concentrating on data loading now. I figured it would be best to get started with the example in [Imports Service Home](https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home) at the "Example Request" heading. I saved the XML for the persons authority records to a file and ran the curl command
:
$ curl -X POST https://cspace.vagrant/cspace-services/imports https://cspace.vagrant/cspace-services/imports
-i -u admin@museumofman.org:Administrator
-H "Content-Type: application/xml"
-T authority-request.xtml
Initially the error response I got back said "Could not process import payload. Check XML markup for not-well-formed errors, elements not matching import schema, etc." and this was logged on the cspace-services.log file:
ERROR [...] [org.collectionspace.services.imports.TemplateExpander:492] ERROR calling expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName for Authority (too few tokens)
So I removed the "<personauthorities_common:refName>" element in the XML file and tried again. The response I got back (with prettified XML) was:
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Mon, 12 Oct 2015 20:05:02 GMT
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=15768000
Content-Type: application/xml
Content-Length: 998
Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9; Path=/cspace-services/; HttpOnly
Vary: Accept-Encoding
<?xml version="1.0" encoding="UTF-8"?>
<import>
<msg>SUCCESS</msg>
<importedRecords>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
</importedRecord>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>11111111-2222-3333-4444-123456789012</csid>
</importedRecord>
</importedRecords>
<status>Success</status>
<totalRecordsImported>2</totalRecordsImported>
<numRecordsImportedByDocType>
<numRecordsImported>
<docType>Personauthority</docType>
<numRecords>2</numRecords>
</numRecordsImported>
</numRecordsImportedByDocType>
<report>READ: /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ: /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
</import>
...which looks like a successful import, BUT this is in cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
...AND the document doesn't seem to be in the database. Based on the csid
I would expect these URLs to work:
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person
...because the documents can't be found (also from cspace-services.log):
ERROR [...] [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303] Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5. Caught exception:Failed to get document /sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
ERROR [...] [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303] Could not find document with CSID=11111111-2222-3333-4444-123456789012. Caught exception:Failed to get document /sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
... and the records don't show up in a search. I feel like I'm missing something here, but I can't quite put my finger on it. Suggestions?
Peter
--
Peter Murray
Dev/Ops Lead and Project Manager
Cherry Hill Company
Success! POSTing to the Import Service with 'service="Persons"' and including the persons_common:inAuthority element worked. I can see the record both in the CSpace web UI and by calling for the record XML directly. I've updated the Gist one more time with the import file that worked in case anyone else stumbles across this in the Talk list archives:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33/d510914e521058e4ac453932bf75fa2b89b2ac0f#file-authority-request-xml <https://gist.github.com/dltj/cacd7dbf011e58d4aa33/d510914e521058e4ac453932bf75fa2b89b2ac0f#file-authority-request-xml>
The documentation did throw me for a bit of a loop -- I thought the two examples were related in that they both created records instead of a record and a container. (And now I'm starting to wonder if the containers are just a special kind of record in the Nuxeo database...)
In addition to the Jira ticket for the documentation, I think there is also one to be made to check for CSpace-specific integrity constraints in the Import service. Without 'inAuthority' the Import service was reporting <msg>SUCCESS</msg> but there was still a problem with the input record. The record might have been okay from a Nuxeo perspective, but from a CSpace perspective the Person record just have that 'inAuthority' element. [[CSPACE-6831] Import service does not check for inAuthority element when adding authority records - CollectionSpace](https://issues.collectionspace.org/browse/CSPACE-6831) If anyone can improve on the description, please do so.
Thank you Ray, John, Aron and Susan. It does take a village to build a CSpace instance.
Peter
> On Oct 13, 2015, at 5:15 PM, Aron Roberts <aron@socrates.berkeley.edu> wrote:
>
> Hey Peter,
>
> Ray and I just talked about this. This is what we *think* we figured out.
>
> - There's definitely confusion over the difference between Personauthority and Person records.
>
> - Yes, Person records now have a repeatable term name group. As John just noted :), the Gist that he helpfully created to paste in a Person import example for PAHMA from "a few years ago" likely used an older schema, one that pre-dated changes introduced c. mid-2012 in CollectionSpace version 2.4, per https://issues.collectionspace.org/browse/CSPACE-4996 <https://issues.collectionspace.org/browse/CSPACE-4996>.
>
> - The selection of an example for creating Personauthority records, in that Imports Service Home document, might be more confusing than helpful ... particularly as the example doesn't include any person records that would then populate that authority with terms.
>
> Generally, we're thinking one might now create Person authorities simply by editing App layer config files and re-initializing authorities, although as an alternative it's still possible to use the imports service for that task. In any event, I'll make a documentation JIRA issue to fix up the Imports Service Home document to reduce confusion, however we end up fixing it.
>
> - Ray's point about Person records needing to point to their parent Personauthoriy, via setting the values of their persons_common:inAuthority fields to the CSID of that parent Personauthority record, is an important one: you'll definitely need to do that when creating Person records that you want to import into a particular authority.
>
> I hope I've got all this more or less right. Please feel free, John and Ray, to jump in as you see fit ...
>
> Aron
>
> On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee <rhlee@berkeley.edu <mailto:rhlee@berkeley.edu>> wrote:
> Peter,
> The examples are indeed out of date. In older versions of cspace displayName was a top-level field, but now it is in the repeating group.
>
> To follow up on Aron's reply, the field that connects a person to a personauthority is persons_common:inAuthority. You must set that to the csid of the personauthority in which the person item should reside (e.g. Local vs. ULAN). That could explain why your record isn't appearing.
>
> Ray
>
>
>
> On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts <aron@socrates.berkeley.edu <mailto:aron@socrates.berkeley.edu>> wrote:
> Peter writes:
> > I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file
>
> True. Personauthorities are authorities for (aka controlled lists of) person terms. Persons are individual terms ("items") in those lists. The former can be conceived of as a container or bucket, for holding terms related to a particular category of persons, and the latter as those person terms that fill it.
>
> If a specific person authority doesn't yet exist, you'd need to create it before you could add terms to it. In the default configuration of CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one sample person authority is created, as specified in config files in the Application layer, via the process of visiting the URL for initializing authorities. Your own tenant's authorities might differ from those in the sample tenants.
>
> It might be helpful to start out with a different import, in part because this one doesn't introduce any of the potential complexities regarding authorities and authority items. Does the Object Exit example in that Imports Service Home document work when you try it, for starters?
>
> https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest <https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest>
>
> That is, if you copy that XML snippet to a different file and use 'curl' with a command pretty much identical to the one you used for personauthorities?
>
>
>
> On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray <pmurray@chillco.com <mailto:pmurray@chillco.com>> wrote:
> Thank you, John -- I hope that will be helpful. One thing I noticed right off the bat is the `service` and `type` attributes on the `import` element:
>
> <import service="Persons" type="Person" CSID="75d9ceef-5ee5-4afe-93da-16cf34700427">
>
> versus
>
> <import seq="2" service="Personauthorities" type="Personauthority" CSID="11111111-2222-3333-4444-123456789012">
>
> I'm not quite sure what the difference between "Persons" and "Personauthorities" is, but there are indeed two different service bindings in my tenant bindings file (reading from https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. <https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues.> -- which itself seems outdated because I didn't find the file where the documentation says the file would be).
>
> Another difference is in the record structure itself. I think you might be on an older version of CSpace because in my persons_common schema the Display Name is in a repeated group and has a slightly different element name:
>
> <persons_common:displayName>G. D. Baron [Ute]</persons_common:displayName>
>
> versus
>
>
> <persons_common:personTermGroupList>
> <persons_common:personTermGroup>
> <persons_common:termDisplayName>Perf Test Person Auth ${docID}</persons_common:termDisplayName>
> </persons_common:personTermGroup>
> </persons_common:personTermGroupList>
>
> I found the latter by looking at what a record looks like in its raw form and comparing it with the documentation. Unfortunately, these changes don't seem to be enough to get the record imported in the right form.
>
>
> Peter
>
>
>> On Oct 13, 2015, at 11:47 AM, John B. LOWE <jblowe@berkeley.edu <mailto:jblowe@berkeley.edu>> wrote:
>>
>> Peter,
>>
>> There are a number of different "styles" or "dialects" of XML that will work with the IMPORT service, it was interesting to see yours.
>>
>> I had a quick look and I noticed your IMPORT document (authority-request.xml) seems to have two instances of the <personauthorities_common:shortIdentifier> element and I'm not sure how that can possibly work.
>>
>> For comparison, I created a gist with 2 Person Authority records and 2 Relations records of the sort we used for the Hearst Museum ("PAHMA") a few years ago. (For reasons I won't get into here, the examples do not use the "sparse payload" approach -- they have lots of empty fields.)
>>
>> https://gist.github.com/jblowe/2a99906195667c27a725 <https://gist.github.com/jblowe/2a99906195667c27a725>
>>
>> But hopefully they will give you the flavor of "our" dialect of IMPORT xml...
>>
>> Good hunting!
>>
>> John
>>
>> On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray <pmurray@chillco.com <mailto:pmurray@chillco.com>> wrote:
>> I'm no further along, but I think I understand the pieces a little better. I put the 'refName' element back in and changed the text of the element to match what I think is our "Local Names" persons authority. I haven't modified the instances of the person authority, so it is using the one defined in base-authority-person.xml (https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46 <https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46>). My request document looks like this:
>>
>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml <https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml>
>>
>> The response from CSpace looks like this:
>>
>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml <https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml>
>>
>> ...and I went looking for the document referenced in the CSpace response; it looks like this:
>>
>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml <https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml>
>>
>> When I look for the record in the UI (https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person <https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person>), it is not there. The only thing in the logs continues to be these warnings in cspace-services.log:
>>
>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
>>
>> The 'catalina.out' file also has these two lines, along with log lines that state the request directory and the document directory are being created in $CATALINA_HOME/temp
>>
>> I have a feeling that there is still something missing in the import request document, but I can't put my finger on it.
>>
>>
>> Peter
>>
>>> On Oct 12, 2015, at 5:52 PM, Aron Roberts <aron@socrates.berkeley.edu <mailto:aron@socrates.berkeley.edu>> wrote:
>>>
>>> At a brief and superficial glance, it seems possible the refName values, in that example on the Imports Service Home page might be obsolete.
>>>
>>> The RefName page (https://wiki.collectionspace.org/display/DOC/RefName <https://wiki.collectionspace.org/display/DOC/RefName>) gives this example, for importing a person authority record:
>>>
>>> urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local Person Vocabulary'
>>>
>>> Note that unlike the example on the Imports Service Home page, that example includes a 'name' component. (As well, the URL in this example also reflects the tenant domain name, which is current practice, IIRC.) Contrast the (presumably correct) example above with this one, from the imports page:
>>>
>>> urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf Test Person Auth'
>>>
>>> Perhaps try putting back the "<personauthorities_common:refName>" element and using the RefName page's example (the first one above) - tweaked as needed to reflect the Museum of Man's values - and see if the import works?
>>>
>>> This might or might not be relevant, but it's what first comes to mind.
>>>
>>> On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray <pmurray@chillco.com <mailto:pmurray@chillco.com>> wrote:
>>> I think I've gotten all of the CSpace configuration and UI changes behind me. Just a few things left to clean up. I'm concentrating on data loading now. I figured it would be best to get started with the example in [Imports Service Home](https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home <https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home>) at the "Example Request" heading. I saved the XML for the persons authority records to a file and ran the `curl command`:
>>>
>>> $ curl -X POST https://cspace.vagrant/cspace-services/imports <https://cspace.vagrant/cspace-services/imports> \
>>> -i -u admin@museumofman.org:Administrator \
>>> -H "Content-Type: application/xml" \
>>> -T authority-request.xtml
>>>
>>> Initially the error response I got back said "Could not process import payload. Check XML markup for not-well-formed errors, elements not matching import schema, etc." and this was logged on the cspace-services.log file:
>>>
>>> ERROR [...] [org.collectionspace.services.imports.TemplateExpander:492] ERROR calling expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName for Authority (too few tokens)
>>>
>>> So I removed the "<personauthorities_common:refName>" element in the XML file and tried again. The response I got back (with prettified XML) was:
>>>
>>> HTTP/1.1 100 Continue
>>>
>>> HTTP/1.1 200 OK
>>> Date: Mon, 12 Oct 2015 20:05:02 GMT
>>> Server: Apache-Coyote/1.1
>>> Strict-Transport-Security: max-age=15768000
>>> Content-Type: application/xml
>>> Content-Length: 998
>>> Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9; Path=/cspace-services/; HttpOnly
>>> Vary: Accept-Encoding
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <import>
>>> <msg>SUCCESS</msg>
>>> <importedRecords>
>>> <importedRecord>
>>> <doctype>Personauthority</doctype>
>>> <csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
>>> </importedRecord>
>>> <importedRecord>
>>> <doctype>Personauthority</doctype>
>>> <csid>11111111-2222-3333-4444-123456789012</csid>
>>> </importedRecord>
>>> </importedRecords>
>>> <status>Success</status>
>>> <totalRecordsImported>2</totalRecordsImported>
>>> <numRecordsImportedByDocType>
>>> <numRecordsImported>
>>> <docType>Personauthority</docType>
>>> <numRecords>2</numRecords>
>>> </numRecordsImported>
>>> </numRecordsImportedByDocType>
>>> <report>READ: /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ: /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
>>> </import>
>>>
>>>
>>> ...which looks like a successful import, BUT this is in cspace-services.log:
>>>
>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no ID for document, won't add org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
>>>
>>>
>>> ...AND the document doesn't seem to be in the database. Based on the `csid` I would expect these URLs to work:
>>>
>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person <https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person>
>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person <https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person>
>>>
>>> ...because the documents can't be found (also from cspace-services.log):
>>>
>>> ERROR [...] [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303] Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5. Caught exception:Failed to get document /sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
>>> ERROR [...] [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303] Could not find document with CSID=11111111-2222-3333-4444-123456789012. Caught exception:Failed to get document /sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
>>>
>>>
>>> ... and the records don't show up in a search. I feel like I'm missing something here, but I can't quite put my finger on it. Suggestions?
>>>
>>>
>>> Peter
--
Peter Murray
Dev/Ops Lead and Project Manager
Cherry Hill Company
AR
Aron Roberts
Tue, Oct 13, 2015 10:48 PM
And now I'm starting to wonder if the containers are just a special kind
of record in the Nuxeo database...
AFAIK:
-
Authority records and the authority term/item record they contain are
both standard kinds of Nuxeo records (probably inheriting from its Document
record type?).
-
Their container/items relationship is maintained solely in CSpace code.
When referring to authorities serving as containers or buckets for items, I
likely was thinking of that latter sense.
Really glad the Person import went well! And certainly, at this point, when
customizing an instance of CSpace for a museum or other collecting
institution, we all benefit greatly from having a village available :).
We're hoping to continue to make this endeavor easier, more intuitive, and
better documented with time, but the value of having that community support
will likely never go away.
On Tue, Oct 13, 2015 at 3:28 PM, Peter Murray pmurray@chillco.com wrote:
Success! POSTing to the Import Service with 'service="Persons"' and
including the persons_common:inAuthority element worked. I can see the
record both in the CSpace web UI and by calling for the record XML
directly. I've updated the Gist one more time with the import file that
worked in case anyone else stumbles across this in the Talk list archives:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33/d510914e521058e4ac453932bf75fa2b89b2ac0f#file-authority-request-xml
The documentation did throw me for a bit of a loop -- I thought the two
examples were related in that they both created records instead of a record
and a container. (And now I'm starting to wonder if the containers are
just a special kind of record in the Nuxeo database...)
In addition to the Jira ticket for the documentation, I think there is
also one to be made to check for CSpace-specific integrity constraints in
the Import service. Without 'inAuthority' the Import service was reporting
<msg>SUCCESS</msg> but there was still a problem with the input record.
The record might have been okay from a Nuxeo perspective, but from a CSpace
perspective the Person record just have that 'inAuthority' element.
[CSPACE-6831] Import service does not check for inAuthority element when
adding authority records - CollectionSpace If anyone can
improve on the description, please do so.
Thank you Ray, John, Aron and Susan. It does take a village to build a
CSpace instance.
Peter
On Oct 13, 2015, at 5:15 PM, Aron Roberts aron@socrates.berkeley.edu
wrote:
Hey Peter,
Ray and I just talked about this. This is what we think we figured out.
-
There's definitely confusion over the difference between
Personauthority and Person records.
-
Yes, Person records now have a repeatable term name group. As John
just noted :), the Gist that he helpfully created to paste in a Person
import example for PAHMA from "a few years ago" likely used an older
schema, one that pre-dated changes introduced c. mid-2012 in
CollectionSpace version 2.4, per
https://issues.collectionspace.org/browse/CSPACE-4996.
-
The selection of an example for creating Personauthority records, in
that Imports Service Home document, might be more confusing than helpful
... particularly as the example doesn't include any person records that
would then populate that authority with terms.
Generally, we're thinking one might now create Person authorities simply
by editing App layer config files and re-initializing authorities, although
as an alternative it's still possible to use the imports service for that
task. In any event, I'll make a documentation JIRA issue to fix up the
Imports Service Home document to reduce confusion, however we end up fixing
it.
- Ray's point about Person records needing to point to their parent
Personauthoriy, via setting the values of their persons_common:inAuthority
fields to the CSID of that parent Personauthority record, is an important
one: you'll definitely need to do that when creating Person records that
you want to import into a particular authority.
I hope I've got all this more or less right. Please feel free, John and
Ray, to jump in as you see fit ...
Aron
On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee rhlee@berkeley.edu wrote:
Peter,
The examples are indeed out of date. In older versions of cspace
displayName was a top-level field, but now it is in the repeating group.
To follow up on Aron's reply, the field that connects a person to a
personauthority is persons_common:inAuthority. You must set that to the
csid of the personauthority in which the person item should reside (e.g.
Local vs. ULAN). That could explain why your record isn't appearing.
Ray
On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts <aron@socrates.berkeley.edu
I'm not quite sure what the difference between "Persons" and
"Personauthorities" is, but there are indeed two different service bindings
in my tenant bindings file
True. Personauthorities are authorities for (aka controlled lists of)
person terms. Persons are individual terms ("items") in those lists. The
former can be conceived of as a container or bucket, for holding terms
related to a particular category of persons, and the latter as those person
terms that fill it.
If a specific person authority doesn't yet exist, you'd need to create
it before you could add terms to it. In the default configuration of
CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one
sample person authority is created, as specified in config files in the
Application layer, via the process of visiting the URL for initializing
authorities. Your own tenant's authorities might differ from those in the
sample tenants.
It might be helpful to start out with a different import, in part
because this one doesn't introduce any of the potential complexities
regarding authorities and authority items. Does the Object Exit example in
that Imports Service Home document work when you try it, for starters?
https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest
That is, if you copy that XML snippet to a different file and use 'curl'
with a command pretty much identical to the one you used for
personauthorities?
On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray pmurray@chillco.com
wrote:
Thank you, John -- I hope that will be helpful. One thing I noticed
right off the bat is the service
and type
attributes on the import
element:
<import service="Persons" type="Person" CSID="
75d9ceef-5ee5-4afe-93da-16cf34700427">
versus
<import seq="2" service="Personauthorities" type="Personauthority" CSID
="11111111-2222-3333-4444-123456789012">
I'm not quite sure what the difference between "Persons" and
"Personauthorities" is, but there are indeed two different service bindings
in my tenant bindings file (reading from
https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. --
which itself seems outdated because I didn't find the file where the
documentation says the file would be).
Another difference is in the record structure itself. I think you
might be on an older version of CSpace because in my persons_common schema
the Display Name is in a repeated group and has a slightly different
element name:
<persons_common:displayName>G. D. Baron [Ute]</persons_common:
displayName>
versus
<persons_common:personTermGroupList><persons_common:personTermGroup> <
persons_common:termDisplayName>Perf Test Person Auth ${docID}</
persons_common:termDisplayName></persons_common:personTermGroup></
persons_common:personTermGroupList>
I found the latter by looking at what a record looks like in its raw
form and comparing it with the documentation. Unfortunately, these changes
don't seem to be enough to get the record imported in the right form.
Peter
On Oct 13, 2015, at 11:47 AM, John B. LOWE jblowe@berkeley.edu wrote:
Peter,
There are a number of different "styles" or "dialects" of XML that will
work with the IMPORT service, it was interesting to see yours.
I had a quick look and I noticed your IMPORT document
(authority-request.xml) seems to have two instances of the <
personauthorities_common:shortIdentifier> element and I'm not sure how
that can possibly work.
For comparison, I created a gist with 2 Person Authority records and 2
Relations records of the sort we used for the Hearst Museum ("PAHMA") a few
years ago. (For reasons I won't get into here, the examples do not use the
"sparse payload" approach -- they have lots of empty fields.)
https://gist.github.com/jblowe/2a99906195667c27a725
But hopefully they will give you the flavor of "our" dialect of IMPORT
xml...
Good hunting!
John
On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray pmurray@chillco.com
wrote:
I'm no further along, but I think I understand the pieces a little
better. I put the 'refName' element back in and changed the text of the
element to match what I think is our "Local Names" persons authority. I
haven't modified the instances of the person authority, so it is using the
one defined in base-authority-person.xml (
https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46).
My request document looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml
The response from CSpace looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml
...and I went looking for the document referenced in the CSpace
response; it looks like this:
https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml
When I look for the record in the UI (
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person),
it is not there. The only thing in the logs continues to be these warnings
in cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
The 'catalina.out' file also has these two lines, along with log lines
that state the request directory and the document directory are being
created in $CATALINA_HOME/temp
I have a feeling that there is still something missing in the import
request document, but I can't put my finger on it.
Peter
On Oct 12, 2015, at 5:52 PM, Aron Roberts aron@socrates.berkeley.edu
wrote:
At a brief and superficial glance, it seems possible the refName
values, in that example on the Imports Service Home page might be obsolete.
The RefName page (
https://wiki.collectionspace.org/display/DOC/RefName) gives this
example, for importing a person authority record:
urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local
Person Vocabulary'
Note that unlike the example on the Imports Service Home page, that
example includes a 'name' component. (As well, the URL in this example also
reflects the tenant domain name, which is current practice, IIRC.) Contrast
the (presumably correct) example above with this one, from the imports page:
urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf
Test Person Auth'
Perhaps try putting back the "<personauthorities_common:refName>"
element and using the RefName page's example (the first one above) -
tweaked as needed to reflect the Museum of Man's values - and see if the
import works?
This might or might not be relevant, but it's what first comes to
mind.
On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray pmurray@chillco.com
wrote:
I think I've gotten all of the CSpace configuration and UI changes
behind me. Just a few things left to clean up. I'm concentrating on data
loading now. I figured it would be best to get started with the example in
Imports Service Home
at the "Example Request" heading. I saved the XML for the persons
authority records to a file and ran the curl command
:
$ curl -X POST https://cspace.vagrant/cspace-services/imports
-i -u admin@museumofman.org:Administrator
-H "Content-Type: application/xml"
-T authority-request.xtml
Initially the error response I got back said "Could not process
import payload. Check XML markup for not-well-formed errors, elements not
matching import schema, etc." and this was logged on the
cspace-services.log file:
ERROR [...]
[org.collectionspace.services.imports.TemplateExpander:492] ERROR calling
expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName
for Authority (too few tokens)
So I removed the "<personauthorities_common:refName>" element in the
XML file and tried again. The response I got back (with prettified XML)
was:
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Mon, 12 Oct 2015 20:05:02 GMT
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=15768000
Content-Type: application/xml
Content-Length: 998
Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9;
Path=/cspace-services/; HttpOnly
Vary: Accept-Encoding
<?xml version="1.0" encoding="UTF-8"?>
<import>
<msg>SUCCESS</msg>
<importedRecords>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
</importedRecord>
<importedRecord>
<doctype>Personauthority</doctype>
<csid>11111111-2222-3333-4444-123456789012</csid>
</importedRecord>
</importedRecords>
<status>Success</status>
<totalRecordsImported>2</totalRecordsImported>
<numRecordsImportedByDocType>
<numRecordsImported>
<docType>Personauthority</docType>
<numRecords>2</numRecords>
</numRecordsImported>
</numRecordsImportedByDocType>
<report>READ:
/home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ:
/home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
</import>
...which looks like a successful import, BUT this is in
cspace-services.log:
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57]
no ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57]
no ID for document, won't add
org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
...AND the document doesn't seem to be in the database. Based on the
csid
I would expect these URLs to work:
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person
https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person
...because the documents can't be found (also from
cspace-services.log):
ERROR [...]
[org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5.
Caught exception:Failed to get document
/sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
ERROR [...]
[org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
Could not find document with CSID=11111111-2222-3333-4444-123456789012.
Caught exception:Failed to get document
/sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
... and the records don't show up in a search. I feel like I'm
missing something here, but I can't quite put my finger on it. Suggestions?
Peter
> And now I'm starting to wonder if the containers are just a special kind
of record in the Nuxeo database...
AFAIK:
* Authority records and the authority term/item record they contain are
both standard kinds of Nuxeo records (probably inheriting from its Document
record type?).
* Their container/items relationship is maintained solely in CSpace code.
When referring to authorities serving as containers or buckets for items, I
likely was thinking of that latter sense.
Really glad the Person import went well! And certainly, at this point, when
customizing an instance of CSpace for a museum or other collecting
institution, we all benefit greatly from having a village available :).
We're hoping to continue to make this endeavor easier, more intuitive, and
better documented with time, but the value of having that community support
will likely never go away.
On Tue, Oct 13, 2015 at 3:28 PM, Peter Murray <pmurray@chillco.com> wrote:
> Success! POSTing to the Import Service with 'service="Persons"' and
> including the persons_common:inAuthority element worked. I can see the
> record both in the CSpace web UI and by calling for the record XML
> directly. I've updated the Gist one more time with the import file that
> worked in case anyone else stumbles across this in the Talk list archives:
>
>
> https://gist.github.com/dltj/cacd7dbf011e58d4aa33/d510914e521058e4ac453932bf75fa2b89b2ac0f#file-authority-request-xml
>
> The documentation did throw me for a bit of a loop -- I thought the two
> examples were related in that they both created records instead of a record
> and a container. (And now I'm starting to wonder if the containers are
> just a special kind of record in the Nuxeo database...)
>
> In addition to the Jira ticket for the documentation, I think there is
> also one to be made to check for CSpace-specific integrity constraints in
> the Import service. Without 'inAuthority' the Import service was reporting
> <msg>SUCCESS</msg> but there was still a problem with the input record.
> The record might have been okay from a Nuxeo perspective, but from a CSpace
> perspective the Person record just have that 'inAuthority' element.
> [[CSPACE-6831] Import service does not check for inAuthority element when
> adding authority records - CollectionSpace](
> https://issues.collectionspace.org/browse/CSPACE-6831) If anyone can
> improve on the description, please do so.
>
> Thank you Ray, John, Aron and Susan. It does take a village to build a
> CSpace instance.
>
>
> Peter
>
> On Oct 13, 2015, at 5:15 PM, Aron Roberts <aron@socrates.berkeley.edu>
> wrote:
>
> Hey Peter,
>
> Ray and I just talked about this. This is what we *think* we figured out.
>
> - There's definitely confusion over the difference between
> Personauthority and Person records.
>
> - Yes, Person records now have a repeatable term name group. As John
> just noted :), the Gist that he helpfully created to paste in a Person
> import example for PAHMA from "a few years ago" likely used an older
> schema, one that pre-dated changes introduced c. mid-2012 in
> CollectionSpace version 2.4, per
> https://issues.collectionspace.org/browse/CSPACE-4996.
>
> - The selection of an example for creating Personauthority records, in
> that Imports Service Home document, might be more confusing than helpful
> ... particularly as the example doesn't include any person records that
> would then populate that authority with terms.
>
> Generally, we're thinking one might now create Person authorities simply
> by editing App layer config files and re-initializing authorities, although
> as an alternative it's still possible to use the imports service for that
> task. In any event, I'll make a documentation JIRA issue to fix up the
> Imports Service Home document to reduce confusion, however we end up fixing
> it.
>
> - Ray's point about Person records needing to point to their parent
> Personauthoriy, via setting the values of their persons_common:inAuthority
> fields to the CSID of that parent Personauthority record, is an important
> one: you'll definitely need to do that when creating Person records that
> you want to import into a particular authority.
>
> I hope I've got all this more or less right. Please feel free, John and
> Ray, to jump in as you see fit ...
>
> Aron
>
> On Tue, Oct 13, 2015 at 1:54 PM, Ray Lee <rhlee@berkeley.edu> wrote:
>
>> Peter,
>> The examples are indeed out of date. In older versions of cspace
>> displayName was a top-level field, but now it is in the repeating group.
>>
>> To follow up on Aron's reply, the field that connects a person to a
>> personauthority is persons_common:inAuthority. You must set that to the
>> csid of the personauthority in which the person item should reside (e.g.
>> Local vs. ULAN). That could explain why your record isn't appearing.
>>
>> Ray
>>
>>
>>
>> On Tue, Oct 13, 2015 at 1:45 PM, Aron Roberts <aron@socrates.berkeley.edu
>> > wrote:
>>
>>> Peter writes:
>>> > I'm not quite sure what the difference between "Persons" and
>>> "Personauthorities" is, but there are indeed two different service bindings
>>> in my tenant bindings file
>>>
>>> True. Personauthorities are authorities for (aka controlled lists of)
>>> person terms. Persons are individual terms ("items") in those lists. The
>>> former can be conceived of as a container or bucket, for holding terms
>>> related to a particular category of persons, and the latter as those person
>>> terms that fill it.
>>>
>>> If a specific person authority doesn't yet exist, you'd need to create
>>> it before you could add terms to it. In the default configuration of
>>> CollectionSpace's demonstration 'core' and 'lifesci' tenants, at least one
>>> sample person authority is created, as specified in config files in the
>>> Application layer, via the process of visiting the URL for initializing
>>> authorities. Your own tenant's authorities might differ from those in the
>>> sample tenants.
>>>
>>> It might be helpful to start out with a different import, in part
>>> because this one doesn't introduce any of the potential complexities
>>> regarding authorities and authority items. Does the Object Exit example in
>>> that Imports Service Home document work when you try it, for starters?
>>>
>>>
>>> https://wiki.collectionspace.org/display/DOC/Imports+Service+Home#ImportsServiceHome-ExampleRequest
>>>
>>> That is, if you copy that XML snippet to a different file and use 'curl'
>>> with a command pretty much identical to the one you used for
>>> personauthorities?
>>>
>>>
>>>
>>> On Tue, Oct 13, 2015 at 11:32 AM, Peter Murray <pmurray@chillco.com>
>>> wrote:
>>>
>>>> Thank you, John -- I hope that will be helpful. One thing I noticed
>>>> right off the bat is the `service` and `type` attributes on the `import`
>>>> element:
>>>>
>>>> <import service="Persons" type="Person" CSID="
>>>> 75d9ceef-5ee5-4afe-93da-16cf34700427">
>>>>
>>>> versus
>>>>
>>>> <import seq="2" service="Personauthorities" type="Personauthority" CSID
>>>> ="11111111-2222-3333-4444-123456789012">
>>>> I'm not quite sure what the difference between "Persons" and
>>>> "Personauthorities" is, but there are indeed two different service bindings
>>>> in my tenant bindings file (reading from
>>>> https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home#ImportsServiceHome-Howtofindtheserviceandtypevalues. --
>>>> which itself seems outdated because I didn't find the file where the
>>>> documentation says the file would be).
>>>>
>>>> Another difference is in the record structure itself. I think you
>>>> might be on an older version of CSpace because in my persons_common schema
>>>> the Display Name is in a repeated group and has a slightly different
>>>> element name:
>>>>
>>>> <persons_common:displayName>G. D. Baron [Ute]</persons_common:
>>>> displayName>
>>>>
>>>> versus
>>>>
>>>>
>>>> <persons_common:personTermGroupList><persons_common:personTermGroup> <
>>>> persons_common:termDisplayName>Perf Test Person Auth ${docID}</
>>>> persons_common:termDisplayName></persons_common:personTermGroup></
>>>> persons_common:personTermGroupList>
>>>> I found the latter by looking at what a record looks like in its raw
>>>> form and comparing it with the documentation. Unfortunately, these changes
>>>> don't seem to be enough to get the record imported in the right form.
>>>>
>>>>
>>>> Peter
>>>>
>>>>
>>>> On Oct 13, 2015, at 11:47 AM, John B. LOWE <jblowe@berkeley.edu> wrote:
>>>>
>>>> Peter,
>>>>
>>>> There are a number of different "styles" or "dialects" of XML that will
>>>> work with the IMPORT service, it was interesting to see yours.
>>>>
>>>> I had a quick look and I noticed your IMPORT document
>>>> (authority-request.xml) seems to have two instances of the <
>>>> personauthorities_common:shortIdentifier> element and I'm not sure how
>>>> that can possibly work.
>>>>
>>>> For comparison, I created a gist with 2 Person Authority records and 2
>>>> Relations records of the sort we used for the Hearst Museum ("PAHMA") a few
>>>> years ago. (For reasons I won't get into here, the examples do not use the
>>>> "sparse payload" approach -- they have lots of empty fields.)
>>>>
>>>> https://gist.github.com/jblowe/2a99906195667c27a725
>>>>
>>>> But hopefully they will give you the flavor of "our" dialect of IMPORT
>>>> xml...
>>>>
>>>> Good hunting!
>>>>
>>>> John
>>>>
>>>> On Tue, Oct 13, 2015 at 8:14 AM, Peter Murray <pmurray@chillco.com>
>>>> wrote:
>>>>
>>>>> I'm no further along, but I think I understand the pieces a little
>>>>> better. I put the 'refName' element back in and changed the text of the
>>>>> element to match what I think is our "Local Names" persons authority. I
>>>>> haven't modified the instances of the person authority, so it is using the
>>>>> one defined in base-authority-person.xml (
>>>>> https://github.com/collectionspace/application/blob/v4.2/tomcat-main/src/main/resources/defaults/base-authority-person.xml#L42-L46).
>>>>> My request document looks like this:
>>>>>
>>>>>
>>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-authority-request-xml
>>>>>
>>>>> The response from CSpace looks like this:
>>>>>
>>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-post-body-xml
>>>>>
>>>>> ...and I went looking for the document referenced in the CSpace
>>>>> response; it looks like this:
>>>>>
>>>>>
>>>>> https://gist.github.com/dltj/cacd7dbf011e58d4aa33#file-11111111-2222-3333-4444-123456789012-document-xml
>>>>>
>>>>> When I look for the record in the UI (
>>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person),
>>>>> it is not there. The only thing in the logs continues to be these warnings
>>>>> in cspace-services.log:
>>>>>
>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>>> ID for document, won't add
>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@7192126e
>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57] no
>>>>> ID for document, won't add
>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@78598d48
>>>>>
>>>>> The 'catalina.out' file also has these two lines, along with log lines
>>>>> that state the request directory and the document directory are being
>>>>> created in $CATALINA_HOME/temp
>>>>>
>>>>> I have a feeling that there is still something missing in the import
>>>>> request document, but I can't put my finger on it.
>>>>>
>>>>>
>>>>> Peter
>>>>>
>>>>> On Oct 12, 2015, at 5:52 PM, Aron Roberts <aron@socrates.berkeley.edu>
>>>>> wrote:
>>>>>
>>>>> At a brief and superficial glance, it seems possible the refName
>>>>> values, in that example on the Imports Service Home page might be obsolete.
>>>>>
>>>>> The RefName page (
>>>>> https://wiki.collectionspace.org/display/DOC/RefName) gives this
>>>>> example, for importing a person authority record:
>>>>>
>>>>> urn:cspace:core.collectionspace.org:personauthorities:name(localPersonVocabulary)'Local
>>>>> Person Vocabulary'
>>>>>
>>>>> Note that unlike the example on the Imports Service Home page, that
>>>>> example includes a 'name' component. (As well, the URL in this example also
>>>>> reflects the tenant domain name, which is current practice, IIRC.) Contrast
>>>>> the (presumably correct) example above with this one, from the imports page:
>>>>>
>>>>> urn:cspace:collectionspace.org:Personauthorities(perfTestPersons)'Perf
>>>>> Test Person Auth'
>>>>>
>>>>> Perhaps try putting back the "<personauthorities_common:refName>"
>>>>> element and using the RefName page's example (the first one above) -
>>>>> tweaked as needed to reflect the Museum of Man's values - and see if the
>>>>> import works?
>>>>>
>>>>> This might or might not be relevant, but it's what first comes to
>>>>> mind.
>>>>>
>>>>> On Mon, Oct 12, 2015 at 1:19 PM, Peter Murray <pmurray@chillco.com>
>>>>> wrote:
>>>>>
>>>>>> I think I've gotten all of the CSpace configuration and UI changes
>>>>>> behind me. Just a few things left to clean up. I'm concentrating on data
>>>>>> loading now. I figured it would be best to get started with the example in
>>>>>> [Imports Service Home](
>>>>>> https://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home)
>>>>>> at the "Example Request" heading. I saved the XML for the persons
>>>>>> authority records to a file and ran the `curl command`:
>>>>>>
>>>>>> $ curl -X POST https://cspace.vagrant/cspace-services/imports \
>>>>>> -i -u admin@museumofman.org:Administrator \
>>>>>> -H "Content-Type: application/xml" \
>>>>>> -T authority-request.xtml
>>>>>>
>>>>>> Initially the error response I got back said "Could not process
>>>>>> import payload. Check XML markup for not-well-formed errors, elements not
>>>>>> matching import schema, etc." and this was logged on the
>>>>>> cspace-services.log file:
>>>>>>
>>>>>> ERROR [...]
>>>>>> [org.collectionspace.services.imports.TemplateExpander:492] ERROR calling
>>>>>> expandXmlPayloadToDirjava.lang.IllegalArgumentException: Malformed refName
>>>>>> for Authority (too few tokens)
>>>>>>
>>>>>> So I removed the "<personauthorities_common:refName>" element in the
>>>>>> XML file and tried again. The response I got back (with prettified XML)
>>>>>> was:
>>>>>>
>>>>>> HTTP/1.1 100 Continue
>>>>>>
>>>>>> HTTP/1.1 200 OK
>>>>>> Date: Mon, 12 Oct 2015 20:05:02 GMT
>>>>>> Server: Apache-Coyote/1.1
>>>>>> Strict-Transport-Security: max-age=15768000
>>>>>> Content-Type: application/xml
>>>>>> Content-Length: 998
>>>>>> Set-Cookie: JSESSIONID=9EB6F3E6A8679B82E55B8E742CA331E9;
>>>>>> Path=/cspace-services/; HttpOnly
>>>>>> Vary: Accept-Encoding
>>>>>>
>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>> <import>
>>>>>> <msg>SUCCESS</msg>
>>>>>> <importedRecords>
>>>>>> <importedRecord>
>>>>>> <doctype>Personauthority</doctype>
>>>>>> <csid>c955a218-6fea-44bd-9dfe-1e7804be7ac5</csid>
>>>>>> </importedRecord>
>>>>>> <importedRecord>
>>>>>> <doctype>Personauthority</doctype>
>>>>>> <csid>11111111-2222-3333-4444-123456789012</csid>
>>>>>> </importedRecord>
>>>>>> </importedRecords>
>>>>>> <status>Success</status>
>>>>>> <totalRecordsImported>2</totalRecordsImported>
>>>>>> <numRecordsImportedByDocType>
>>>>>> <numRecordsImported>
>>>>>> <docType>Personauthority</docType>
>>>>>> <numRecords>2</numRecords>
>>>>>> </numRecordsImported>
>>>>>> </numRecordsImportedByDocType>
>>>>>> <report>READ:
>>>>>> /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/11111111-2222-3333-4444-123456789012/document.xml/Personauthorities/11111111-2222-3333-4444-123456789012READ:
>>>>>> /home/vagrant/cspace-tomcat/apache-tomcat-7.0.57/temp/imports-2086091545258611644/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5/document.xml/Personauthorities/c955a218-6fea-44bd-9dfe-1e7804be7ac5</report>
>>>>>> </import>
>>>>>>
>>>>>>
>>>>>> ...which looks like a successful import, BUT this is in
>>>>>> cspace-services.log:
>>>>>>
>>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57]
>>>>>> no ID for document, won't add
>>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@74ae85f4
>>>>>> WARN [...] [org.nuxeo.ecm.core.io.impl.AbstractDocumentReader:57]
>>>>>> no ID for document, won't add
>>>>>> org.nuxeo.ecm.core.io.impl.ExportedDocumentImpl@3f4d7382
>>>>>>
>>>>>>
>>>>>> ...AND the document doesn't seem to be in the database. Based on the
>>>>>> `csid` I would expect these URLs to work:
>>>>>>
>>>>>>
>>>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=c955a218-6fea-44bd-9dfe-1e7804be7ac5&vocab=person
>>>>>>
>>>>>> https://cspace.vagrant/collectionspace/ui/sdmom/html/person.html?csid=11111111-2222-3333-4444-123456789012&vocab=person
>>>>>>
>>>>>> ...because the documents can't be found (also from
>>>>>> cspace-services.log):
>>>>>>
>>>>>> ERROR [...]
>>>>>> [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
>>>>>> Could not find document with CSID=c955a218-6fea-44bd-9dfe-1e7804be7ac5.
>>>>>> Caught exception:Failed to get document
>>>>>> /sdmom-domain/Workspaces/Persons/c955a218-6fea-44bd-9dfe-1e7804be7ac5
>>>>>> ERROR [...]
>>>>>> [org.collectionspace.services.nuxeo.client.java.RepositoryJavaClientImpl:1303]
>>>>>> Could not find document with CSID=11111111-2222-3333-4444-123456789012.
>>>>>> Caught exception:Failed to get document
>>>>>> /sdmom-domain/Workspaces/Persons/11111111-2222-3333-4444-123456789012
>>>>>>
>>>>>>
>>>>>> ... and the records don't show up in a search. I feel like I'm
>>>>>> missing something here, but I can't quite put my finger on it. Suggestions?
>>>>>>
>>>>>>
>>>>>> Peter
>>>>>
>>>>>
> --
> Peter Murray
> Dev/Ops Lead and Project Manager
> Cherry Hill Company
>
>
>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
>
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>
>