talk@lists.collectionspace.org

WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org

View all threads

CSpace 4.3 -- data seems to have disappeared on production. In effort to replace a user, says UserID exists

AR
Aron Roberts
Thu, Jun 16, 2016 8:24 PM

Thanks, Ray!

Have just created this bug; please feel free to comment/annotate as you see
fit:

https://issues.collectionspace.org/browse/CSPACE-6964

On Thu, Jun 16, 2016 at 1:00 PM, Ray Lee rhlee@berkeley.edu wrote:

My observation has been that if you're using instance-qualified databases, ant
create_db -Drecreate_db=true does not delete them. I think that's a bug,
since cspace now defaults to using instance-qualified databases. To start
fresh, you have to issue DROP DATABASE commands to delete all the cspace
databases. The names of the databases depend on the instance id you have
configured, but by default they'll be cspace_default, nuxeo_default,
fcart_default, etc. Given all the problems you're having, I would also
rebuild/redeploy the services, application, and ui layers.

Ray

On Thu, Jun 16, 2016 at 10:29 AM, James Keeline james@keeline.com wrote:

I am coordinating with our sysadmin on this.

I just saw this.  He and I had been conferring and were going to restore
a DB backup from a week or so ago.  This would be followed by a CSpace *shutdown.sh -force* and a *startup.sh* and probably the "initialise" and "init"
URL calls after login.


However, Aron, I see your recommendation that we run the command again
from ~/cspace_source/services/

ant undeploy deploy create_db -Drecreate_db=true import

Thanks for jumping in to help again.  Once we have a stable install
again, we can recreate data done manually and reload the person and object
values via import services.

James D. Keeline


From: Aron Roberts aron@socrates.berkeley.edu
To: James Keeline james@keeline.com
Cc: Christopher R. HOFFMAN chris_h@berkeley.edu; Talk <
talk@lists.collectionspace.org>
Sent: Thursday, June 16, 2016 9:58 AM

Subject: Re: [Talk] CSpace 4.3 -- data seems to have disappeared on
production. In effort to replace a user, says UserID exists

Hi James,

So, before we get into the tedious work, I want to know how we may have

a clean slate from which to work for this install.

Running 'ant undeploy deploy create_db -Drecreate_db=true import' once
again, from within the topmost directory of the source code tree for the
CollectionSpace Services layer, should do what you're asking for here, IIRC.

I used psql to delete the two user account rows in accounts_tenants

and accounts_common.  However, the web UI will still not allow me to add
the user accounts.

Data connected with user accounts are present in multiple related tables
in CollectionSpace's AuthN/AuthZ database, beyond even those two mentioned
tables. So wiping everything, as mentioned above, is the way to go.

Aron

On Thu, Jun 16, 2016 at 9:54 AM, Aron Roberts <aron@socrates.berkeley.edu

wrote:

He mentioned running an ant script as part of a process to fix the

login issue.

ant undeploy deploy create_db -Drecreate_db=true import

Running the Ant 'create_db' target with the '-Drecreate_db=true' flag
drops and re-creates databases, which would perfectly explain the missing
data :(

More on this flag:
https://issues.collectionspace.org/browse/CSPACE-6322

AFAIK, the CollectionSpace documentation spotlights the destructive
nature of this command, wherever it's mentioned. E.g. in
https://wiki.collectionspace.org/display/DOC/Installing+4.3+on+Ubuntu+14.04
:

[image: Inline image 1]
If the requisite user accounts are once again working, after this command
was run, reloading data might be the next step.

Aron

P.S. ***** Note to the project team: It looks like that warning needs to
be revised, however, to make it even clearer that all relevant databases -
that hold AuthN/AuthZ data, as well as all records and relationship data
pertinent to a CollectionSpace installation, across all tenants - are
dropped. That behavior might have been implicit in the current wording,
when it was originally written - back then, the 'cspace' database held auth
data and the 'nuxeo' database held all other data - but is no longer so.


On Thu, Jun 16, 2016 at 7:15 AM, James Keeline james@keeline.com wrote:

Seeing none of the data expected other than user accounts that don't show
up in the web UI, I was going to get started on recreating the manual
data.  I used psql to delete the two user account rows in accounts_tenants
and accounts_common.  However, the web UI will still not allow me to add
the user accounts.  This message is shown at the bottom of the browser
screen:

POST failed: create in
org.collectionspace.services.account.AccountResourceUserId exists. Non
unique userId=tiffany@sdchm.org

So, before we get into the tedious work, I want to know how we may have a
clean slate from which to work for this install.  This is our production
server and we are previewing it to the director before next week's
scheduled training.  On our side we'd like to get this done soon.

Thank you for the help so far.

James D. Keeline


From: James Keeline james@keeline.com
To: Christopher R. HOFFMAN chris_h@berkeley.edu; Aron Roberts <
aron@socrates.berkeley.edu>
Cc: Talk talk@lists.collectionspace.org
Sent: Wednesday, June 15, 2016 9:50 PM

Subject: Re: [Talk] CSpace 4.3 -- data seems to have disappeared on
production. In effort to replace a user, says UserID exists

That may indeed be the way to go.  I went through Aron's suggestions and
the postgres tables are largely empty.  The exception is the
accounts_common which did include the two additions even though not
displayed in the web interface.

One can certainly spend too much time trying to understand what went
wrong and/or fixing it when the best course of action may be to clear the
databases and start again.

Is there a document that describes this process?  (Truncating or
dropping tables or dropping entire databases and installing fresh).

We have most of the files (except my customizations to the form which I
was installing this morning) in a git repository.  I would imagine that a
fresh install of some kind will cause some confusion with that.

Aside from my installing files in the application and ui layer to
customize the form, the other thing that our sysadmin did was assign a
domain name to the IP.  He mentioned running an ant script as part of a
process to fix the login issue.

ant undeploy deploy create_db -Drecreate_db=true import

Looking at the history, there were a number of commands that might purge
the system.

However, I'm more interested in a path forward that will let us move
towards deployment and training for the museum.

I appreciate the help and suggestions.

James D. Keeline


From: Christopher R. HOFFMAN chris_h@berkeley.edu
To: James Keeline james@keeline.com
Cc: Talk talk@lists.collectionspace.org
Sent: Wednesday, June 15, 2016 9:27 PM
Subject: Re: [Talk] CSpace 4.3 -- data seems to have disappeared on
production. In effort to replace a user, says UserID exists

Very strange, James. I would recommend stepping back and then doing some
things like checking the URLs that you are using. Maybe you are trying to
log into the core tenant rather than your museum tenant?

Otherwise yes I would say remove the database and move on.

Good luck, we are looking forward to hearing how it goes.

Chris

On Jun 15, 2016, at 4:42 PM, James Keeline james@keeline.com wrote:

I am bringing over the data from my local VirtualBox install over to our
production system which was upgraded to 4.3.

On production I manually created:

- 2 user accounts for login
- 62 storage locations (described previously)
- 18 or so dynasties

I also used the import service to load the person records (artists,
authors, employees).  That was done from an XML file and is repeatable
easily when needed.

As we were preparing to preview it to the museum director before next
week's training, I checked each of the above and found them present and
pointed out what they should look for in terms of the customized form,
select lists, and term lists.  It worked this morning.

This afternoon my PM said she could not log in as the director's account
created and the data was missing.  When I visited myself, the site had many
appearances of a new install.

I had resigned myself to having to create those manual values again,
reimport the people, and this time make sure we had a DB backup for future
restores.

After putting out some other fires, I began to create the first user
account.  I was pretty surprised to see this error at the bottom:

POST failed: create in
org.collectionspace.services.account.AccountResourceUserId exists. Non
unique userId=tiffany@sdchm.org

So, this implies that my data is there and I might be able to restore
it.  I'd appreciate suggestions on this.

Sidebars of what can cause this are helpful too.

If I need to wipe the DB and start from scratch then I need that so we
can go forward with recreating the data and loading the object data which a
colleague has been working on (with some help from me).  He is having some
success on his local install so I think we are making headway there.  There
are some things I'm not sure about (CSpace object IDs to be assigned) but I
need to focus on the above issue to meet our timeline.

Thank you again for any help or comments you can provide.

James D. Keeline


Talk mailing list
Talk@lists.collectionspace.org

http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org


Talk mailing list
Talk@lists.collectionspace.org

http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org


Talk mailing list
Talk@lists.collectionspace.org

http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org

Thanks, Ray! Have just created this bug; please feel free to comment/annotate as you see fit: https://issues.collectionspace.org/browse/CSPACE-6964 On Thu, Jun 16, 2016 at 1:00 PM, Ray Lee <rhlee@berkeley.edu> wrote: > My observation has been that if you're using instance-qualified databases, ant > create_db -Drecreate_db=true does not delete them. I think that's a bug, > since cspace now defaults to using instance-qualified databases. To start > fresh, you have to issue DROP DATABASE commands to delete all the cspace > databases. The names of the databases depend on the instance id you have > configured, but by default they'll be cspace_default, nuxeo_default, > fcart_default, etc. Given all the problems you're having, I would also > rebuild/redeploy the services, application, and ui layers. > > Ray > > On Thu, Jun 16, 2016 at 10:29 AM, James Keeline <james@keeline.com> wrote: > >> I am coordinating with our sysadmin on this. >> >> I just saw this. He and I had been conferring and were going to restore >> a DB backup from a week or so ago. This would be followed by a CSpace `*shutdown.sh >> -force*` and a `*startup.sh*` and probably the "*initialise*" and "*init*" >> URL calls after login. >> ____ >> >> However, Aron, I see your recommendation that we run the command again >> from ~/cspace_source/services/ >> >> ant undeploy deploy create_db -Drecreate_db=true import >> >> Thanks for jumping in to help again. Once we have a stable install >> again, we can recreate data done manually and reload the person and object >> values via import services. >> >> James D. Keeline >> >> ------------------------------ >> *From:* Aron Roberts <aron@socrates.berkeley.edu> >> *To:* James Keeline <james@keeline.com> >> *Cc:* Christopher R. HOFFMAN <chris_h@berkeley.edu>; Talk < >> talk@lists.collectionspace.org> >> *Sent:* Thursday, June 16, 2016 9:58 AM >> >> *Subject:* Re: [Talk] CSpace 4.3 -- data seems to have disappeared on >> production. In effort to replace a user, says UserID exists >> >> Hi James, >> >> > So, before we get into the tedious work, I want to know how we may have >> a clean slate from which to work for this install. >> >> Running 'ant undeploy deploy create_db -Drecreate_db=true import' once >> again, from within the topmost directory of the source code tree for the >> CollectionSpace Services layer, should do what you're asking for here, IIRC. >> >> > I used psql to delete the two user account rows in accounts_tenants >> and accounts_common. However, the web UI will still not allow me to add >> the user accounts. >> >> Data connected with user accounts are present in multiple related tables >> in CollectionSpace's AuthN/AuthZ database, beyond even those two mentioned >> tables. So wiping everything, as mentioned above, is the way to go. >> >> Aron >> >> On Thu, Jun 16, 2016 at 9:54 AM, Aron Roberts <aron@socrates.berkeley.edu >> > wrote: >> >> > He mentioned running an ant script as part of a process to fix the >> login issue. >> > >> > ant undeploy deploy create_db -Drecreate_db=true import >> >> Running the Ant 'create_db' target with the '-Drecreate_db=true' flag >> drops and re-creates databases, which would perfectly explain the missing >> data :( >> >> More on this flag: >> https://issues.collectionspace.org/browse/CSPACE-6322 >> >> AFAIK, the CollectionSpace documentation spotlights the destructive >> nature of this command, wherever it's mentioned. E.g. in >> https://wiki.collectionspace.org/display/DOC/Installing+4.3+on+Ubuntu+14.04 >> : >> >> [image: Inline image 1] >> If the requisite user accounts are once again working, after this command >> was run, reloading data might be the next step. >> >> Aron >> >> P.S. ***** Note to the project team: It looks like that warning needs to >> be revised, however, to make it even clearer that all relevant databases - >> that hold AuthN/AuthZ data, as well as all records and relationship data >> pertinent to a CollectionSpace installation, across all tenants - are >> dropped. That behavior might have been implicit in the current wording, >> when it was originally written - back then, the 'cspace' database held auth >> data and the 'nuxeo' database held all other data - but is no longer so. >> ***** >> >> >> On Thu, Jun 16, 2016 at 7:15 AM, James Keeline <james@keeline.com> wrote: >> >> Seeing none of the data expected other than user accounts that don't show >> up in the web UI, I was going to get started on recreating the manual >> data. I used psql to delete the two user account rows in accounts_tenants >> and accounts_common. However, the web UI will still not allow me to add >> the user accounts. This message is shown at the bottom of the browser >> screen: >> >> POST failed: create in >> org.collectionspace.services.account.AccountResourceUserId exists. Non >> unique userId=tiffany@sdchm.org >> >> So, before we get into the tedious work, I want to know how we may have a >> clean slate from which to work for this install. This is our production >> server and we are previewing it to the director before next week's >> scheduled training. On our side we'd like to get this done soon. >> >> Thank you for the help so far. >> >> James D. Keeline >> >> ------------------------------ >> *From:* James Keeline <james@keeline.com> >> *To:* Christopher R. HOFFMAN <chris_h@berkeley.edu>; Aron Roberts < >> aron@socrates.berkeley.edu> >> *Cc:* Talk <talk@lists.collectionspace.org> >> *Sent:* Wednesday, June 15, 2016 9:50 PM >> >> *Subject:* Re: [Talk] CSpace 4.3 -- data seems to have disappeared on >> production. In effort to replace a user, says UserID exists >> >> That may indeed be the way to go. I went through Aron's suggestions and >> the postgres tables are largely empty. The exception is the >> accounts_common which did include the two additions even though not >> displayed in the web interface. >> >> One can certainly spend too much time trying to understand what went >> wrong and/or fixing it when the best course of action may be to clear the >> databases and start again. >> >> *Is there a document that describes this process? (Truncating or >> dropping tables or dropping entire databases and installing fresh).* >> >> We have most of the files (except my customizations to the form which I >> was installing this morning) in a git repository. I would imagine that a >> fresh install of some kind will cause some confusion with that. >> >> Aside from my installing files in the application and ui layer to >> customize the form, the other thing that our sysadmin did was assign a >> domain name to the IP. He mentioned running an ant script as part of a >> process to fix the login issue. >> >> *ant undeploy deploy create_db -Drecreate_db=true import* >> >> Looking at the history, there were a number of commands that might purge >> the system. >> >> However, I'm more interested in a path forward that will let us move >> towards deployment and training for the museum. >> >> I appreciate the help and suggestions. >> >> James D. Keeline >> >> ------------------------------ >> *From:* Christopher R. HOFFMAN <chris_h@berkeley.edu> >> *To:* James Keeline <james@keeline.com> >> *Cc:* Talk <talk@lists.collectionspace.org> >> *Sent:* Wednesday, June 15, 2016 9:27 PM >> *Subject:* Re: [Talk] CSpace 4.3 -- data seems to have disappeared on >> production. In effort to replace a user, says UserID exists >> >> Very strange, James. I would recommend stepping back and then doing some >> things like checking the URLs that you are using. Maybe you are trying to >> log into the core tenant rather than your museum tenant? >> >> Otherwise yes I would say remove the database and move on. >> >> Good luck, we are looking forward to hearing how it goes. >> >> Chris >> >> On Jun 15, 2016, at 4:42 PM, James Keeline <james@keeline.com> wrote: >> >> I am bringing over the data from my local VirtualBox install over to our >> production system which was upgraded to 4.3. >> >> On production I manually created: >> >> >> - 2 user accounts for login >> - 62 storage locations (described previously) >> - 18 or so dynasties >> >> >> I also used the import service to load the person records (artists, >> authors, employees). That was done from an XML file and is repeatable >> easily when needed. >> >> As we were preparing to preview it to the museum director before next >> week's training, I checked each of the above and found them present and >> pointed out what they should look for in terms of the customized form, >> select lists, and term lists. It worked this morning. >> >> This afternoon my PM said she could not log in as the director's account >> created and the data was missing. When I visited myself, the site had many >> appearances of a new install. >> >> I had resigned myself to having to create those manual values again, >> reimport the people, and this time make sure we had a DB backup for future >> restores. >> >> After putting out some other fires, I began to create the first user >> account. I was pretty surprised to see this error at the bottom: >> >> POST failed: create in >> org.collectionspace.services.account.AccountResourceUserId exists. Non >> unique userId=tiffany@sdchm.org >> >> So, this implies that my data is there and I might be able to restore >> it. I'd appreciate suggestions on this. >> >> Sidebars of what can cause this are helpful too. >> >> If I need to wipe the DB and start from scratch then I need that so we >> can go forward with recreating the data and loading the object data which a >> colleague has been working on (with some help from me). He is having some >> success on his local install so I think we are making headway there. There >> are some things I'm not sure about (CSpace object IDs to be assigned) but I >> need to focus on the above issue to meet our timeline. >> >> Thank you again for any help or comments you can provide. >> >> James D. Keeline >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> >> >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> >> >> >> >> >> >> _______________________________________________ >> Talk mailing list >> Talk@lists.collectionspace.org >> >> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org >> >> >