talk@lists.collectionspace.org

WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org

View all threads

Import: "succeeds", but database is empty

NS
Nate Solas
Fri, Mar 23, 2012 7:12 PM

Even better: list the commits. I'm working on creating a git remote that
points to the official collectionspace/services code (somehow we didn't
fork that when we start, but we'll fix that later), and I'm going to cherry
pick the commits. This is a much more scalable approach than listing files,
since for instance the very latest commit
to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the
one before that does.

I can start on the list of hashes once I get going?
Nate

On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez jmartinez@movingimage.uswrote:

I think it would be useful to list these patches for 2.0 on the Deployment
wiki space. MMI is using the 2.0 code base too until we adopt 2.3/2.4. Or
maybe create a JIRA filter for these patched and unpatched issues and link
it to the same page.

  • Jesse

On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee rhlee@berkeley.edu wrote:

The relationship problem Chris mentioned is CSPACE-4783, and can be
patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in the
relations service with the 2.2 version.

I should also mention that while I said there were four bugs relating to
import in 2.0, there are actually more. For example, CSPACE-4789 is still
floating around. But those four were the ones we ran into most often.

Ray

On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote:

Glad to hear it's working!!  We're actually using 2.0 for several major
installations right now, but we do have it patched, and patched, and
patched…  The demo.collectionspace.org site is not patched and has some
significant bugs for instance with saving of relationships.  That's a big
one.  You want to make sure you have that patch installed.

We will probably wait for 2.3 to do the next set of upgrades.

Chris

On Mar 23, 2012, at 10:41 AM, Nate Solas wrote:

Ray, you the man. Imports seem to work like a charm now.

...Can't believe I didn't find that ticket. Should we not be working with
2.0 at all?

Nate

On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee rhlee@berkeley.edu wrote:

Hi Nate,
On 2.0 there are four bugs related to import, and two files to patch.

CSPACE-4817 is not fixed.
CSPACE-4866 is fixed, in ImportsResource.java
CSPACE-3911 is fixed, in ImportsResource.java
CSPACE-4807 is fixed, in ImportsResource.java and
nuxeo/ImportCommand.java

Basically, if you replace ImportsResource.java and
nuxeo/ImportCommand.java with the latest in master, you'll have the patches.

At that point, you should use the 2.2 syntax (not the 2.1 and below,
because now the system is patched up to 2.2, as far as imports are
concerned). The deprecated syntax won't work reliably until CSPACE-4817 is
fixed.

It sounds like you may have missed patching nuxeo/ImportCommand.java.
I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us
know if that works!

Ray

On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas nate.solas@walkerart.org
wrote:

I'm totally stuck. I've been working with our customized codebase trying
to run imports and have had some extremely weird and frustrating behavior.
I'll run an import of 10 collection objects, everything looks right, but
nothing shows up in the cspace admin. Incredibly, sometimes I will see the
objects in the nuxeo database, but un-recognized by cspace services
apparently. Other times nothing will show up but another import of the same
data will work. I've restarted tomcat between attempts, wped the database,
and tried both the old and new curl import commands (
http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home).
I seemed to have more success with the old, deprecated one, but that was
running into the "fail silently" bug all the time.
I finally installed the patched ImportsResource.java from
http://issues.collectionspace.org/browse/CSPACE-4866 to see if that
helps, and it's the same behavior. I reverted all of our changes and am
running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's
failing every time. Per that ticket I'm trying the dirt-simplest xml import
I can think of, the objectexit one, and... nothing. I've even tried it with
the old syntax and without Aron's patch which didn't work the first time,
then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried
the old one again (curl -X) and it worked! Seeing that things were behaving
I tried my 10 objects again (which HAVE worked at times in the past) and it
failed (silently).
I guess this relates to the recent thread about testing imports. I've
been going nuts for a few days on this, trying to eliminate all the
variables on my end, tweaking xml generators, etc, and now as I read more
it seems like there are just a bunch of known "random glitches" with the
imports?
Any tips? More info below:
curl http://localhost:8180/cspace-services/imports?type=xml -i -u
admin@core.collectionspace.org:Administrator -F "
file=@objectexit-utf8.xml;type=application/xml"
catalina.out looks like:
Created ThreadLocal instance: java.lang.ThreadLocal - null
Setting ThreadLocal instance: java.lang.ThreadLocal -
javax.security.auth.login.LoginContext@1f5173f
Preamble type is:
Media type is:application/xml
############## TEMPLATE_DIR:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates
##############
OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca
====Path============
/imports/import[0]
====Context=========
====Fragment========
http://collectionspace.org/services/objectexit"
name="objectexit_common">
This OBJECT has been declared a World Heritage Nuisance and must
be disposed of forthwith - Sebastián
OE2010.2

===================
Making directory:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9
====TemplateExpander DONE============

importTree reading file:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca
exists? true
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader ::
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader ::
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml
:: DONE

Even better: list the commits. I'm working on creating a git remote that points to the official collectionspace/services code (somehow we didn't fork that when we start, but we'll fix that later), and I'm going to cherry pick the commits. This is a much more scalable approach than listing files, since for instance the very latest commit to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the one before that does. I can start on the list of hashes once I get going? Nate On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez <jmartinez@movingimage.us>wrote: > I think it would be useful to list these patches for 2.0 on the Deployment > wiki space. MMI is using the 2.0 code base too until we adopt 2.3/2.4. Or > maybe create a JIRA filter for these patched and unpatched issues and link > it to the same page. > > - Jesse > > > On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee <rhlee@berkeley.edu> wrote: > >> The relationship problem Chris mentioned is CSPACE-4783, and can be >> patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in the >> relations service with the 2.2 version. >> >> I should also mention that while I said there were four bugs relating to >> import in 2.0, there are actually more. For example, CSPACE-4789 is still >> floating around. But those four were the ones we ran into most often. >> >> Ray >> >> >> On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote: >> >> Glad to hear it's working!! We're actually using 2.0 for several major >> installations right now, but we do have it patched, and patched, and >> patched… The demo.collectionspace.org site is not patched and has some >> significant bugs for instance with saving of relationships. That's a big >> one. You want to make sure you have that patch installed. >> >> We will probably wait for 2.3 to do the next set of upgrades. >> >> Chris >> >> On Mar 23, 2012, at 10:41 AM, Nate Solas wrote: >> >> Ray, you the man. Imports seem to work like a charm now. >> >> ...Can't believe I didn't find that ticket. Should we not be working with >> 2.0 at all? >> >> Nate >> >> On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee <rhlee@berkeley.edu> wrote: >> >>> Hi Nate, >>> On 2.0 there are four bugs related to import, and two files to patch. >>> >>> CSPACE-4817 is not fixed. >>> CSPACE-4866 is fixed, in ImportsResource.java >>> CSPACE-3911 is fixed, in ImportsResource.java >>> CSPACE-4807 is fixed, in ImportsResource.java and >>> nuxeo/ImportCommand.java >>> >>> Basically, if you replace ImportsResource.java and >>> nuxeo/ImportCommand.java with the latest in master, you'll have the patches. >>> >>> At that point, you should use the 2.2 syntax (not the 2.1 and below, >>> because now the system is patched up to 2.2, as far as imports are >>> concerned). The deprecated syntax won't work reliably until CSPACE-4817 is >>> fixed. >>> >>> It sounds like you may have missed patching nuxeo/ImportCommand.java. >>> I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us >>> know if that works! >>> >>> Ray >>> >>> >>> >>> On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas <nate.solas@walkerart.org> >>> wrote: >>> >>> I'm totally stuck. I've been working with our customized codebase trying >>> to run imports and have had some extremely weird and frustrating behavior. >>> I'll run an import of 10 collection objects, everything looks right, but >>> nothing shows up in the cspace admin. Incredibly, sometimes I will see the >>> objects in the nuxeo database, but un-recognized by cspace services >>> apparently. Other times nothing will show up but another import of the same >>> data will work. I've restarted tomcat between attempts, wped the database, >>> and tried both the old and new curl import commands ( >>> http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home). >>> I seemed to have more success with the old, deprecated one, but that was >>> running into the "fail silently" bug all the time. >>> I finally installed the patched ImportsResource.java from >>> http://issues.collectionspace.org/browse/CSPACE-4866 to see if that >>> helps, and it's the same behavior. I reverted all of our changes and am >>> running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's >>> failing every time. Per that ticket I'm trying the dirt-simplest xml import >>> I can think of, the objectexit one, and... nothing. I've even tried it with >>> the old syntax and without Aron's patch which didn't work the first time, >>> then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried >>> the old one again (curl -X) and it worked! Seeing that things were behaving >>> I tried my 10 objects again (which HAVE worked at times in the past) and it >>> failed (silently). >>> I guess this relates to the recent thread about testing imports. I've >>> been going nuts for a few days on this, trying to eliminate all the >>> variables on my end, tweaking xml generators, etc, and now as I read more >>> it seems like there are just a bunch of known "random glitches" with the >>> imports? >>> Any tips? More info below: >>> curl http://localhost:8180/cspace-services/imports?type=xml -i -u >>> admin@core.collectionspace.org:Administrator -F " >>> file=@objectexit-utf8.xml;type=application/xml" >>> catalina.out looks like: >>> Created ThreadLocal instance: java.lang.ThreadLocal - null >>> Setting ThreadLocal instance: java.lang.ThreadLocal - >>> javax.security.auth.login.LoginContext@1f5173f >>> Preamble type is: >>> Media type is:application/xml >>> ############## TEMPLATE_DIR: >>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates >>> ############## >>> OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca >>> ====Path============ >>> /imports/import[0] >>> ====Context========= >>> ====Fragment======== >>> http://collectionspace.org/services/objectexit" >>> name="objectexit_common"> >>> This OBJECT has been declared a World Heritage Nuisance and must >>> be disposed of forthwith - Sebastián >>> OE2010.2 >>> >>> >>> =================== >>> Making directory: >>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9 >>> ====TemplateExpander DONE============ >>> ================ >>> importTree reading file: >>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca >>> exists? true >>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: >>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml >>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: >>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml >>> :: DONE >>> >>> >>> >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> >> >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org > >
RL
Ray Lee
Fri, Mar 23, 2012 7:35 PM

Even better: list the commits. I'm working on creating a git remote that points to the official collectionspace/services code (somehow we didn't fork that when we start, but we'll fix that later), and I'm going to cherry pick the commits. This is a much more scalable approach than listing files, since for instance the very latest commit to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the one before that does.

I can start on the list of hashes once I get going?
Nate

On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez jmartinez@movingimage.us wrote:
I think it would be useful to list these patches for 2.0 on the Deployment wiki space. MMI is using the 2.0 code base too until we adopt 2.3/2.4. Or maybe create a JIRA filter for these patched and unpatched issues and link it to the same page.

  • Jesse

On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee rhlee@berkeley.edu wrote:
The relationship problem Chris mentioned is CSPACE-4783, and can be patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in the relations service with the 2.2 version.

I should also mention that while I said there were four bugs relating to import in 2.0, there are actually more. For example, CSPACE-4789 is still floating around. But those four were the ones we ran into most often.

Ray

On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote:

Glad to hear it's working!!  We're actually using 2.0 for several major installations right now, but we do have it patched, and patched, and patched…  The demo.collectionspace.org site is not patched and has some significant bugs for instance with saving of relationships.  That's a big one.  You want to make sure you have that patch installed.

We will probably wait for 2.3 to do the next set of upgrades.

Chris

On Mar 23, 2012, at 10:41 AM, Nate Solas wrote:

Ray, you the man. Imports seem to work like a charm now.

...Can't believe I didn't find that ticket. Should we not be working with 2.0 at all?

Nate

On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee rhlee@berkeley.edu wrote:
Hi Nate,
On 2.0 there are four bugs related to import, and two files to patch.

CSPACE-4817 is not fixed.
CSPACE-4866 is fixed, in ImportsResource.java
CSPACE-3911 is fixed, in ImportsResource.java
CSPACE-4807 is fixed, in ImportsResource.java and nuxeo/ImportCommand.java

Basically, if you replace ImportsResource.java and nuxeo/ImportCommand.java with the latest in master, you'll have the patches.

At that point, you should use the 2.2 syntax (not the 2.1 and below, because now the system is patched up to 2.2, as far as imports are concerned). The deprecated syntax won't work reliably until CSPACE-4817 is fixed.

It sounds like you may have missed patching nuxeo/ImportCommand.java. I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us know if that works!

Ray

On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas nate.solas@walkerart.org wrote:

I'm totally stuck. I've been working with our customized codebase trying to run imports and have had some extremely weird and frustrating behavior. I'll run an import of 10 collection objects, everything looks right, but nothing shows up in the cspace admin. Incredibly, sometimes I will see the objects in the nuxeo database, but un-recognized by cspace services apparently. Other times nothing will show up but another import of the same data will work. I've restarted tomcat between attempts, wped the database, and tried both the old and new curl import commands (http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home). I seemed to have more success with the old, deprecated one, but that was running into the "fail silently" bug all the time.

I finally installed the patched ImportsResource.java from http://issues.collectionspace.org/browse/CSPACE-4866 to see if that helps, and it's the same behavior. I reverted all of our changes and am running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's failing every time. Per that ticket I'm trying the dirt-simplest xml import I can think of, the objectexit one, and... nothing. I've even tried it with the old syntax and without Aron's patch which didn't work the first time, then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried the old one again (curl -X) and it worked! Seeing that things were behaving I tried my 10 objects again (which HAVE worked at times in the past) and it failed (silently).
I guess this relates to the recent thread about testing imports. I've been going nuts for a few days on this, trying to eliminate all the variables on my end, tweaking xml generators, etc, and now as I read more it seems like there are just a bunch of known "random glitches" with the imports?
Any tips? More info below:
curl http://localhost:8180/cspace-services/imports?type=xml -i -u admin@core.collectionspace.org:Administrator -F "file=@objectexit-utf8.xml;type=application/xml"
catalina.out looks like:
Created ThreadLocal instance: java.lang.ThreadLocal - null
Setting ThreadLocal instance: java.lang.ThreadLocal - javax.security.auth.login.LoginContext@1f5173f
Preamble type is:
Media type is:application/xml
############## TEMPLATE_DIR: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates
############## OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca
====Path============
/imports/import[0]
====Context=========
====Fragment========
http://collectionspace.org/services/objectexit" name="objectexit_common">
This OBJECT has been declared a World Heritage Nuisance and must be disposed of forthwith - Sebastián
OE2010.2

===================
Making directory: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9
====TemplateExpander DONE============

importTree reading file: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca exists? true
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml :: DONE


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

For those two patches I mentioned, I only listed the last commit. I think for cherry picking you need them all, right? Here they are: CSPACE-4843 is actually three commits: https://github.com/collectionspace/application/commit/3f762d943d775d142159d43d2750123ccf19979d https://github.com/collectionspace/application/commit/8ca8c904214e04f13681d787e5efba95b3216293 https://github.com/collectionspace/application/commit/6ca6ee32836b17b11fef283f514b5c854c1bcdeb CSPACE-4721 is two: https://github.com/collectionspace/application/commit/5324138792317a77cc9a4ad4ee3b29d670ed1730 https://github.com/collectionspace/application/commit/4b803f2164eca5230d8ac24554454018ced74e02 Ray On Mar 23, 2012, at 12:12 PM, Nate Solas wrote: > Even better: list the commits. I'm working on creating a git remote that points to the official collectionspace/services code (somehow we didn't fork that when we start, but we'll fix that later), and I'm going to cherry pick the commits. This is a much more scalable approach than listing files, since for instance the very latest commit to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the one before that does. > > I can start on the list of hashes once I get going? > Nate > > > On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez <jmartinez@movingimage.us> wrote: > I think it would be useful to list these patches for 2.0 on the Deployment wiki space. MMI is using the 2.0 code base too until we adopt 2.3/2.4. Or maybe create a JIRA filter for these patched and unpatched issues and link it to the same page. > > - Jesse > > > On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee <rhlee@berkeley.edu> wrote: > The relationship problem Chris mentioned is CSPACE-4783, and can be patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in the relations service with the 2.2 version. > > I should also mention that while I said there were four bugs relating to import in 2.0, there are actually more. For example, CSPACE-4789 is still floating around. But those four were the ones we ran into most often. > > Ray > > > On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote: > >> Glad to hear it's working!! We're actually using 2.0 for several major installations right now, but we do have it patched, and patched, and patched… The demo.collectionspace.org site is not patched and has some significant bugs for instance with saving of relationships. That's a big one. You want to make sure you have that patch installed. >> >> We will probably wait for 2.3 to do the next set of upgrades. >> >> Chris >> >> On Mar 23, 2012, at 10:41 AM, Nate Solas wrote: >> >>> Ray, you the man. Imports seem to work like a charm now. >>> >>> ...Can't believe I didn't find that ticket. Should we not be working with 2.0 at all? >>> >>> Nate >>> >>> On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee <rhlee@berkeley.edu> wrote: >>> Hi Nate, >>> On 2.0 there are four bugs related to import, and two files to patch. >>> >>> CSPACE-4817 is not fixed. >>> CSPACE-4866 is fixed, in ImportsResource.java >>> CSPACE-3911 is fixed, in ImportsResource.java >>> CSPACE-4807 is fixed, in ImportsResource.java and nuxeo/ImportCommand.java >>> >>> Basically, if you replace ImportsResource.java and nuxeo/ImportCommand.java with the latest in master, you'll have the patches. >>> >>> At that point, you should use the 2.2 syntax (not the 2.1 and below, because now the system is patched up to 2.2, as far as imports are concerned). The deprecated syntax won't work reliably until CSPACE-4817 is fixed. >>> >>> It sounds like you may have missed patching nuxeo/ImportCommand.java. I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us know if that works! >>> >>> Ray >>> >>> >>> >>> On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas <nate.solas@walkerart.org> wrote: >>> >>> I'm totally stuck. I've been working with our customized codebase trying to run imports and have had some extremely weird and frustrating behavior. I'll run an import of 10 collection objects, everything looks right, but nothing shows up in the cspace admin. Incredibly, sometimes I will see the objects in the nuxeo database, but un-recognized by cspace services apparently. Other times nothing will show up but another import of the same data will work. I've restarted tomcat between attempts, wped the database, and tried both the old and new curl import commands (http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home). I seemed to have more success with the old, deprecated one, but that was running into the "fail silently" bug all the time. >>> >>> I finally installed the patched ImportsResource.java from http://issues.collectionspace.org/browse/CSPACE-4866 to see if that helps, and it's the same behavior. I reverted all of our changes and am running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's failing every time. Per that ticket I'm trying the dirt-simplest xml import I can think of, the objectexit one, and... nothing. I've even tried it with the old syntax and without Aron's patch which didn't work the first time, then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried the old one again (curl -X) and it worked! Seeing that things were behaving I tried my 10 objects again (which HAVE worked at times in the past) and it failed (silently). >>> I guess this relates to the recent thread about testing imports. I've been going nuts for a few days on this, trying to eliminate all the variables on my end, tweaking xml generators, etc, and now as I read more it seems like there are just a bunch of known "random glitches" with the imports? >>> Any tips? More info below: >>> curl http://localhost:8180/cspace-services/imports?type=xml -i -u admin@core.collectionspace.org:Administrator -F "file=@objectexit-utf8.xml;type=application/xml" >>> catalina.out looks like: >>> Created ThreadLocal instance: java.lang.ThreadLocal - null >>> Setting ThreadLocal instance: java.lang.ThreadLocal - javax.security.auth.login.LoginContext@1f5173f >>> Preamble type is: >>> Media type is:application/xml >>> ############## TEMPLATE_DIR: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates >>> ############## OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca >>> ====Path============ >>> /imports/import[0] >>> ====Context========= >>> ====Fragment======== >>> http://collectionspace.org/services/objectexit" name="objectexit_common"> >>> This OBJECT has been declared a World Heritage Nuisance and must be disposed of forthwith - Sebastián >>> OE2010.2 >>> >>> >>> =================== >>> Making directory: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9 >>> ====TemplateExpander DONE============ >>> ================ >>> importTree reading file: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca exists? true >>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml >>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml :: DONE >>> >>> >>> _______________________________________________ >>> Talk mailing list >>> Talk@lists.collectionspace.org >>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org > > > > _______________________________________________ > Talk mailing list > Talk@lists.collectionspace.org > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org > >
NS
Nate Solas
Fri, Mar 23, 2012 7:50 PM

Good call. I was searching the git commit logs for the JIRA # and saw some
multi-commit patches, but we're not bullding our own app layer yet so I
skipped it. Here's what I have so far, including your latest commits.

Patches needed for 2.0:

Services: (tested and seems good)
de5ba110804ca81b11a263d902b04d2835f87fba CSPACE-4807 -2m
6a06c3070c13a2543aa15d7e35126b847d177abb CSPACE-4783 -12d
a4aaebe3bb5bf4dfe9455eb778806f603d19684b CSPACE-4866 -9d
a30acc72cf5beeb1f164881ed0a4f993a1cd6c51 CSPACE-3911 -4d -- conflict here?
take the changes from 3911

Application: (I haven't tested since we're not building the app layer
in-house yet)
3f762d943d775d142159d43d2750123ccf19979d CSPACE-4843
8ca8c904214e04f13681d787e5efba95b3216293
6ca6ee32836b17b11fef283f514b5c854c1bcdeb
5324138792317a77cc9a4ad4ee3b29d670ed1730 CSPACE-4721
4b803f2164eca5230d8ac24554454018ced74e02

On Fri, Mar 23, 2012 at 2:35 PM, Ray Lee rhlee@berkeley.edu wrote:

For those two patches I mentioned, I only listed the last commit. I think
for cherry picking you need them all, right? Here they are:

CSPACE-4843 is actually three commits:

https://github.com/collectionspace/application/commit/3f762d943d775d142159d43d2750123ccf19979d

https://github.com/collectionspace/application/commit/8ca8c904214e04f13681d787e5efba95b3216293

https://github.com/collectionspace/application/commit/6ca6ee32836b17b11fef283f514b5c854c1bcdeb

CSPACE-4721 is two:

https://github.com/collectionspace/application/commit/5324138792317a77cc9a4ad4ee3b29d670ed1730

https://github.com/collectionspace/application/commit/4b803f2164eca5230d8ac24554454018ced74e02

Ray

On Mar 23, 2012, at 12:12 PM, Nate Solas wrote:

Even better: list the commits. I'm working on creating a git remote that
points to the official collectionspace/services code (somehow we didn't
fork that when we start, but we'll fix that later), and I'm going to cherry
pick the commits. This is a much more scalable approach than listing files,
since for instance the very latest commit
to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the
one before that does.

I can start on the list of hashes once I get going?
Nate

On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez jmartinez@movingimage.uswrote:

I think it would be useful to list these patches for 2.0 on the
Deployment wiki space. MMI is using the 2.0 code base too until we adopt
2.3/2.4. Or maybe create a JIRA filter for these patched and unpatched
issues and link it to the same page.

  • Jesse

On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee rhlee@berkeley.edu wrote:

The relationship problem Chris mentioned is CSPACE-4783, and can be
patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in
the relations service with the 2.2 version.

I should also mention that while I said there were four bugs relating to
import in 2.0, there are actually more. For example, CSPACE-4789 is still
floating around. But those four were the ones we ran into most often.

Ray

On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote:

Glad to hear it's working!!  We're actually using 2.0 for several major
installations right now, but we do have it patched, and patched, and
patched…  The demo.collectionspace.org site is not patched and has some
significant bugs for instance with saving of relationships.  That's a big
one.  You want to make sure you have that patch installed.

We will probably wait for 2.3 to do the next set of upgrades.

Chris

On Mar 23, 2012, at 10:41 AM, Nate Solas wrote:

Ray, you the man. Imports seem to work like a charm now.

...Can't believe I didn't find that ticket. Should we not be working
with 2.0 at all?

Nate

On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee rhlee@berkeley.edu wrote:

Hi Nate,
On 2.0 there are four bugs related to import, and two files to patch.

CSPACE-4817 is not fixed.
CSPACE-4866 is fixed, in ImportsResource.java
CSPACE-3911 is fixed, in ImportsResource.java
CSPACE-4807 is fixed, in ImportsResource.java and
nuxeo/ImportCommand.java

Basically, if you replace ImportsResource.java and
nuxeo/ImportCommand.java with the latest in master, you'll have the patches.

At that point, you should use the 2.2 syntax (not the 2.1 and below,
because now the system is patched up to 2.2, as far as imports are
concerned). The deprecated syntax won't work reliably until CSPACE-4817 is
fixed.

It sounds like you may have missed patching nuxeo/ImportCommand.java.
I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us
know if that works!

Ray

On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas <
nate.solas@walkerart.org> wrote:

I'm totally stuck. I've been working with our customized codebase
trying to run imports and have had some extremely weird and frustrating
behavior. I'll run an import of 10 collection objects, everything looks
right, but nothing shows up in the cspace admin. Incredibly, sometimes I
will see the objects in the nuxeo database, but un-recognized by cspace
services apparently. Other times nothing will show up but another import of
the same data will work. I've restarted tomcat between attempts, wped the
database, and tried both the old and new curl import commands (
http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home).
I seemed to have more success with the old, deprecated one, but that was
running into the "fail silently" bug all the time.
I finally installed the patched ImportsResource.java from
http://issues.collectionspace.org/browse/CSPACE-4866 to see if that
helps, and it's the same behavior. I reverted all of our changes and am
running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's
failing every time. Per that ticket I'm trying the dirt-simplest xml import
I can think of, the objectexit one, and... nothing. I've even tried it with
the old syntax and without Aron's patch which didn't work the first time,
then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried
the old one again (curl -X) and it worked! Seeing that things were behaving
I tried my 10 objects again (which HAVE worked at times in the past) and it
failed (silently).
I guess this relates to the recent thread about testing imports. I've
been going nuts for a few days on this, trying to eliminate all the
variables on my end, tweaking xml generators, etc, and now as I read more
it seems like there are just a bunch of known "random glitches" with the
imports?
Any tips? More info below:
curl http://localhost:8180/cspace-services/imports?type=xml -i -u
admin@core.collectionspace.org:Administrator -F "
file=@objectexit-utf8.xml;type=application/xml"
catalina.out looks like:
Created ThreadLocal instance: java.lang.ThreadLocal - null
Setting ThreadLocal instance: java.lang.ThreadLocal -
javax.security.auth.login.LoginContext@1f5173f
Preamble type is:
Media type is:application/xml
############## TEMPLATE_DIR:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates
##############
OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca
====Path============
/imports/import[0]
====Context=========
====Fragment========
http://collectionspace.org/services/objectexit"
name="objectexit_common">
This OBJECT has been declared a World Heritage Nuisance and must
be disposed of forthwith - Sebastián
OE2010.2

===================
Making directory:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9
====TemplateExpander DONE============

importTree reading file:
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca
exists? true
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader ::
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml
~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader ::
/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml
:: DONE

Good call. I was searching the git commit logs for the JIRA # and saw some multi-commit patches, but we're not bullding our own app layer yet so I skipped it. Here's what I have so far, including your latest commits. Patches needed for 2.0: Services: (tested and seems good) de5ba110804ca81b11a263d902b04d2835f87fba CSPACE-4807 -2m 6a06c3070c13a2543aa15d7e35126b847d177abb CSPACE-4783 -12d a4aaebe3bb5bf4dfe9455eb778806f603d19684b CSPACE-4866 -9d a30acc72cf5beeb1f164881ed0a4f993a1cd6c51 CSPACE-3911 -4d -- conflict here? take the changes from 3911 Application: (I haven't tested since we're not building the app layer in-house yet) 3f762d943d775d142159d43d2750123ccf19979d CSPACE-4843 8ca8c904214e04f13681d787e5efba95b3216293 6ca6ee32836b17b11fef283f514b5c854c1bcdeb 5324138792317a77cc9a4ad4ee3b29d670ed1730 CSPACE-4721 4b803f2164eca5230d8ac24554454018ced74e02 On Fri, Mar 23, 2012 at 2:35 PM, Ray Lee <rhlee@berkeley.edu> wrote: > For those two patches I mentioned, I only listed the last commit. I think > for cherry picking you need them all, right? Here they are: > > CSPACE-4843 is actually three commits: > > > https://github.com/collectionspace/application/commit/3f762d943d775d142159d43d2750123ccf19979d > > https://github.com/collectionspace/application/commit/8ca8c904214e04f13681d787e5efba95b3216293 > > https://github.com/collectionspace/application/commit/6ca6ee32836b17b11fef283f514b5c854c1bcdeb > > CSPACE-4721 is two: > > > https://github.com/collectionspace/application/commit/5324138792317a77cc9a4ad4ee3b29d670ed1730 > > https://github.com/collectionspace/application/commit/4b803f2164eca5230d8ac24554454018ced74e02 > > > Ray > > > On Mar 23, 2012, at 12:12 PM, Nate Solas wrote: > > Even better: list the commits. I'm working on creating a git remote that > points to the official collectionspace/services code (somehow we didn't > fork that when we start, but we'll fix that later), and I'm going to cherry > pick the commits. This is a much more scalable approach than listing files, > since for instance the very latest commit > to nuxeo/RelationDocumentModelHandler.java doesn't compile for me but the > one before that does. > > I can start on the list of hashes once I get going? > Nate > > > On Fri, Mar 23, 2012 at 2:04 PM, Jesse Martinez <jmartinez@movingimage.us>wrote: > >> I think it would be useful to list these patches for 2.0 on the >> Deployment wiki space. MMI is using the 2.0 code base too until we adopt >> 2.3/2.4. Or maybe create a JIRA filter for these patched and unpatched >> issues and link it to the same page. >> >> - Jesse >> >> >> On Fri, Mar 23, 2012 at 2:06 PM, Ray Lee <rhlee@berkeley.edu> wrote: >> >>> The relationship problem Chris mentioned is CSPACE-4783, and can be >>> patched in 2.0/2.1 by replacing RelationDocumentModelHandler.java in >>> the relations service with the 2.2 version. >>> >>> I should also mention that while I said there were four bugs relating to >>> import in 2.0, there are actually more. For example, CSPACE-4789 is still >>> floating around. But those four were the ones we ran into most often. >>> >>> Ray >>> >>> >>> On Mar 23, 2012, at 10:44 AM, Chris Hoffman wrote: >>> >>> Glad to hear it's working!! We're actually using 2.0 for several major >>> installations right now, but we do have it patched, and patched, and >>> patched… The demo.collectionspace.org site is not patched and has some >>> significant bugs for instance with saving of relationships. That's a big >>> one. You want to make sure you have that patch installed. >>> >>> We will probably wait for 2.3 to do the next set of upgrades. >>> >>> Chris >>> >>> On Mar 23, 2012, at 10:41 AM, Nate Solas wrote: >>> >>> Ray, you the man. Imports seem to work like a charm now. >>> >>> ...Can't believe I didn't find that ticket. Should we not be working >>> with 2.0 at all? >>> >>> Nate >>> >>> On Fri, Mar 23, 2012 at 12:12 PM, Ray Lee <rhlee@berkeley.edu> wrote: >>> >>>> Hi Nate, >>>> On 2.0 there are four bugs related to import, and two files to patch. >>>> >>>> CSPACE-4817 is not fixed. >>>> CSPACE-4866 is fixed, in ImportsResource.java >>>> CSPACE-3911 is fixed, in ImportsResource.java >>>> CSPACE-4807 is fixed, in ImportsResource.java and >>>> nuxeo/ImportCommand.java >>>> >>>> Basically, if you replace ImportsResource.java and >>>> nuxeo/ImportCommand.java with the latest in master, you'll have the patches. >>>> >>>> At that point, you should use the 2.2 syntax (not the 2.1 and below, >>>> because now the system is patched up to 2.2, as far as imports are >>>> concerned). The deprecated syntax won't work reliably until CSPACE-4817 is >>>> fixed. >>>> >>>> It sounds like you may have missed patching nuxeo/ImportCommand.java. >>>> I'll link CSPACE-4807 to the other jiras, so it's easier to find. Let us >>>> know if that works! >>>> >>>> Ray >>>> >>>> >>>> >>>> On Fri, 23 Mar 2012 10:59:34 -0500, Nate Solas < >>>> nate.solas@walkerart.org> wrote: >>>> >>>> I'm totally stuck. I've been working with our customized codebase >>>> trying to run imports and have had some extremely weird and frustrating >>>> behavior. I'll run an import of 10 collection objects, everything looks >>>> right, but nothing shows up in the cspace admin. Incredibly, sometimes I >>>> will see the objects in the nuxeo database, but un-recognized by cspace >>>> services apparently. Other times nothing will show up but another import of >>>> the same data will work. I've restarted tomcat between attempts, wped the >>>> database, and tried both the old and new curl import commands ( >>>> http://wiki.collectionspace.org/display/collectionspace/Imports+Service+Home). >>>> I seemed to have more success with the old, deprecated one, but that was >>>> running into the "fail silently" bug all the time. >>>> I finally installed the patched ImportsResource.java from >>>> http://issues.collectionspace.org/browse/CSPACE-4866 to see if that >>>> helps, and it's the same behavior. I reverted all of our changes and am >>>> running a vanilla Collectionspace 2.0 (with Aron's patch), and now it's >>>> failing every time. Per that ticket I'm trying the dirt-simplest xml import >>>> I can think of, the objectexit one, and... nothing. I've even tried it with >>>> the old syntax and without Aron's patch which didn't work the first time, >>>> then I restarted tomcat, tried the new (2.1 and below) syntax: fail. Tried >>>> the old one again (curl -X) and it worked! Seeing that things were behaving >>>> I tried my 10 objects again (which HAVE worked at times in the past) and it >>>> failed (silently). >>>> I guess this relates to the recent thread about testing imports. I've >>>> been going nuts for a few days on this, trying to eliminate all the >>>> variables on my end, tweaking xml generators, etc, and now as I read more >>>> it seems like there are just a bunch of known "random glitches" with the >>>> imports? >>>> Any tips? More info below: >>>> curl http://localhost:8180/cspace-services/imports?type=xml -i -u >>>> admin@core.collectionspace.org:Administrator -F " >>>> file=@objectexit-utf8.xml;type=application/xml" >>>> catalina.out looks like: >>>> Created ThreadLocal instance: java.lang.ThreadLocal - null >>>> Setting ThreadLocal instance: java.lang.ThreadLocal - >>>> javax.security.auth.login.LoginContext@1f5173f >>>> Preamble type is: >>>> Media type is:application/xml >>>> ############## TEMPLATE_DIR: >>>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/cspace/config/services/resources/templates >>>> ############## >>>> OUTPUT_DIR:/home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca >>>> ====Path============ >>>> /imports/import[0] >>>> ====Context========= >>>> ====Fragment======== >>>> http://collectionspace.org/services/objectexit" >>>> name="objectexit_common"> >>>> This OBJECT has been declared a World Heritage Nuisance and must >>>> be disposed of forthwith - Sebastián >>>> OE2010.2 >>>> >>>> >>>> =================== >>>> Making directory: >>>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9 >>>> ====TemplateExpander DONE============ >>>> ================ >>>> importTree reading file: >>>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca >>>> exists? true >>>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: >>>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml >>>> ~~~~~~~~~~~~~~~~~~~ LoggedXMLDirectoryReader :: >>>> /home/nschroeder/cs_2.0/apache-tomcat-6.0.33/temp/imports-f9ea92df-ec33-4dbe-8ba4-e4329974a0ca/ObjectExit/da01e9fb-f6f9-4865-99f1-3ad380d364c9/document.xml >>>> :: DONE >>>> >>>> >>>> >>> >>> _______________________________________________ >>> Talk mailing list >>> Talk@lists.collectionspace.org >>> >>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >>> >>> >>> >>> >>> _______________________________________________ >>> Talk mailing list >>> Talk@lists.collectionspace.org >>> >>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >>> >>> >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> > >