talk@lists.collectionspace.org
WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org
View all threadsFwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is related to another project using the Nuxeo DAMS. ANyone else using Talend out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll share whatever we learn.
Thanks - Patrick
Begin forwarded message:
From: Brian Tingle Brian.Tingle@ucop.edu
Date: December 9, 2013 4:34:26 PM EST
To: "ucldc@googlegroups.com" ucldc@googlegroups.com
Subject: [UCLDC] FW: Talend
See below from Nuxeo. Any thoughts on using a nuxeo API from talend? -- Brian
From: Brendan Coveney [mailto:bcoveney@nuxeo.com]
Sent: Monday, December 09, 2013 1:23 PM
To: Harlan Brown
Cc: Brian Tingle; Adrian Turner; Sherri Berger
Subject: Re: Talend
And just to be clear after speaking to Thierry, he agrees that we "strongly discourage" the SQL injection approach.
We have a customer in France using Talend who use the REST approach and it works really well.
Regards
Brendan
On 9 December 2013 13:08, Harlan Brown hbrown@nuxeo.com wrote:
Our CTO, Thierry Delprat (AKA Tiry) wrote some extensions for Talend that access the Nuxeo automation API.
They're on github: https://github.com/tiry/nuxeo-talend-components
Have a look, try some test transactions, and let us know how it goes.
Thanks
--
Harlan Brown
Technical Account Manager / Technical Consultant
Nuxeo | New York
Tel. (847) 809-1813
@nuxeo | @harlanbrown
--
Brendan Coveney
General Manager Americas and Asia Pacific, Nuxeo
Phone (US): +1.408-5068630
email: bcoveney@nuxeo.com
skype: bcoveney , tweet @bcoveney
--
You received this message because you are subscribed to the Google Groups "UCLDC project team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ucldc+unsubscribe@googlegroups.com.
To post to this group, send email to ucldc@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
All,
The contractor working on the Walker's data migration wants to work directly in SQL. Seeing the note below, combined with aberrant behavior we see when inserting records directly into tables raises concern and gives pause. Do we know why Nuxeo strongly discourages the SQL injection approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz <pschmitz@berkeley.edumailto:pschmitz@berkeley.edu>
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.orgmailto:talk@lists.collectionspace.org List" <talk@lists.collectionspace.orgmailto:talk@lists.collectionspace.org>
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is related to another project using the Nuxeo DAMS. ANyone else using Talend out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll share whatever we learn.
Thanks - Patrick
Begin forwarded message:
From: Brian Tingle <Brian.Tingle@ucop.edumailto:Brian.Tingle@ucop.edu>
Date: December 9, 2013 4:34:26 PM EST
To: "ucldc@googlegroups.commailto:ucldc@googlegroups.com" <ucldc@googlegroups.commailto:ucldc@googlegroups.com>
Subject: [UCLDC] FW: Talend
See below from Nuxeo. Any thoughts on using a nuxeo API from talend? -- Brian
From: Brendan Coveney [mailto:bcoveney@nuxeo.com]
Sent: Monday, December 09, 2013 1:23 PM
To: Harlan Brown
Cc: Brian Tingle; Adrian Turner; Sherri Berger
Subject: Re: Talend
And just to be clear after speaking to Thierry, he agrees that we "strongly discourage" the SQL injection approach.
We have a customer in France using Talend who use the REST approach and it works really well.
Regards
Brendan
[https://app.yesware.com/t/9b25f5f4c12cfd9dfb8338b7bd1a558b9ecedbe2/e38805163f4e231505e82e29501d52ce/spacer.gif][http://app.yesware.com/t/9b25f5f4c12cfd9dfb8338b7bd1a558b9ecedbe2/e38805163f4e231505e82e29501d52ce/spacer.gif]
On 9 December 2013 13:08, Harlan Brown <hbrown@nuxeo.commailto:hbrown@nuxeo.com> wrote:
Our CTO, Thierry Delprat (AKA Tiry) wrote some extensions for Talend that access the Nuxeo automation API.
They're on github: https://github.com/tiry/nuxeo-talend-components
Have a look, try some test transactions, and let us know how it goes.
Thanks
--
Harlan Brown
Technical Account Manager / Technical Consultant
Nuxeo | New York
Tel. (847) 809-1813tel:%28847%29%20809-1813
@nuxeo | @harlanbrown
--
Brendan Coveney
General Manager Americas and Asia Pacific, Nuxeo
Phone (US): +1.408-5068630
email: bcoveney@nuxeo.commailto:bcoveney@nuxeo.com
skype: bcoveney , tweet @bcoveney
--
You received this message because you are subscribed to the Google Groups "UCLDC project team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ucldc+unsubscribe@googlegroups.commailto:ucldc+unsubscribe@googlegroups.com.
To post to this group, send email to ucldc@googlegroups.commailto:ucldc@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying attachments) is confidential and may contain privileged or proprietary information that is exempt from disclosure under applicable law. If you are not the intended recipient, or believe that you have received this message in error, you are strictly prohibited from disclosing, copying, distributing, or using the information contained herein. Inadvertent or unauthorized disclosure shall not compromise or waive the confidentiality of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately notify us at maildisc@walkerart.org and delete this transmission from your computer.
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that UCLDC conversation. However, I can say that we decided not to use straight SQL for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting on top of the Nuxeo ECM platform. Records are not linked together by the kinds of patterns that normally apply. The "id" fields in each table are not what connects them -- you have to create records in the hierarchy table too. This pattern is used even for repeating fields within a record type, usually that is!
b) CollectionSpace itself then does some interesting things, like creating records in the misc table (that might be a nuxeo thing, I don't know) and collectionspace_core. There is more here that I don't understand.
c) To support search, Nuxeo also populates a fulltext index. This is not done by database triggers, so the index is only updated by Nuxeo. We're working on a way to re-index the fulltext index because we've been using SQL to update some records.
I'd definitely be interested to hear if others have figured out how to do migrations using straight SQL, but we've been importing XML records. We're happy to share our techniques and code with your contractor. It's all up in Jira, the wiki, and even github.
Regards,
Chris
On Dec 19, 2013, at 8:33 AM, Dowden, Robin wrote:
All,
The contractor working on the Walker's data migration wants to work directly in SQL. Seeing the note below, combined with aberrant behavior we see when inserting records directly into tables raises concern and gives pause. Do we know why Nuxeo strongly discourages the SQL injection approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz pschmitz@berkeley.edu
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.org List" talk@lists.collectionspace.org
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is related to another project using the Nuxeo DAMS. ANyone else using Talend out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll share whatever we learn.
Thanks - Patrick
Begin forwarded message:
From: Brian Tingle Brian.Tingle@ucop.edu
Date: December 9, 2013 4:34:26 PM EST
To: "ucldc@googlegroups.com" ucldc@googlegroups.com
Subject: [UCLDC] FW: Talend
See below from Nuxeo. Any thoughts on using a nuxeo API from talend? -- Brian
From: Brendan Coveney [mailto:bcoveney@nuxeo.com]
Sent: Monday, December 09, 2013 1:23 PM
To: Harlan Brown
Cc: Brian Tingle; Adrian Turner; Sherri Berger
Subject: Re: Talend
And just to be clear after speaking to Thierry, he agrees that we "strongly discourage" the SQL injection approach.
We have a customer in France using Talend who use the REST approach and it works really well.
Regards
Brendan
On 9 December 2013 13:08, Harlan Brown hbrown@nuxeo.com wrote:
Our CTO, Thierry Delprat (AKA Tiry) wrote some extensions for Talend that access the Nuxeo automation API.
They're on github: https://github.com/tiry/nuxeo-talend-components
Have a look, try some test transactions, and let us know how it goes.
Thanks
--
Harlan Brown
Technical Account Manager / Technical Consultant
Nuxeo | New York
Tel. (847) 809-1813
@nuxeo | @harlanbrown
--
Brendan Coveney
General Manager Americas and Asia Pacific, Nuxeo
Phone (US): +1.408-5068630
email: bcoveney@nuxeo.com
skype: bcoveney , tweet @bcoveney
--
You received this message because you are subscribed to the Google Groups "UCLDC project team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ucldc+unsubscribe@googlegroups.com.
To post to this group, send email to ucldc@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying attachments) is confidential and may contain privileged or proprietary information that is exempt from disclosure under applicable law. If you are not the intended recipient, or believe that you have received this message in error, you are strictly prohibited from disclosing, copying, distributing, or using the information contained herein. Inadvertent or unauthorized disclosure shall not compromise or waive the confidentiality of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately notify us at maildisc@walkerart.org and delete this transmission from your computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Hi all,
How many (if any) of the problems caused by using SQL might be avoided if
NXQL were used instead? Presumably, using NXQL would make Nuxeo "aware" of
the operations being performed, but that's just supposition on my part. If
this were the case, then perhaps at least the index-not-updating issue
might be avoided?
I imagine that if any of the processes that Chris pointed out (e.g.,
creating/updating records in misc and collectionspace_core) were handled by
CSpace services, rather than Nuxeo, the NXQL approach wouldn't be any
better than SQL for those aspects.
Of course, using any *QL approach relies on the user knowing the exact
structure of the underlying DB, which can often (but by no means always) be
predicted given the fairly consistent patterns used.
Michael
On Thu, Dec 19, 2013 at 8:57 AM, Chris Hoffman chris_h@berkeley.edu wrote:
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that UCLDC
conversation. However, I can say that we decided not to use straight SQL
for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting on
top of the Nuxeo ECM platform. Records are not linked together by the
kinds of patterns that normally apply. The "id" fields in each table are
not what connects them -- you have to create records in the hierarchy table
too. This pattern is used even for repeating fields within a record type,
usually that is!
b) CollectionSpace itself then does some interesting things, like creating
records in the misc table (that might be a nuxeo thing, I don't know) and
collectionspace_core. There is more here that I don't understand.
c) To support search, Nuxeo also populates a fulltext index. This is not
done by database triggers, so the index is only updated by Nuxeo. We're
working on a way to re-index the fulltext index because we've been using
SQL to update some records.
I'd definitely be interested to hear if others have figured out how to do
migrations using straight SQL, but we've been importing XML records. We're
happy to share our techniques and code with your contractor. It's all up
in Jira, the wiki, and even github.
Regards,
Chris
On Dec 19, 2013, at 8:33 AM, Dowden, Robin wrote:
All,
The contractor working on the Walker's data migration wants to work
directly in SQL. Seeing the note below, combined with aberrant behavior we
see when inserting records directly into tables raises concern and gives
pause. Do we know why Nuxeo strongly discourages the SQL injection
approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz pschmitz@berkeley.edu
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.org List" talk@lists.collectionspace.org
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is
related to another project using the Nuxeo DAMS. ANyone else using Talend
out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll
share whatever we learn.
Thanks - Patrick
Begin forwarded message:
*From: *Brian Tingle Brian.Tingle@ucop.edu
*Date: *December 9, 2013 4:34:26 PM EST
*To: *"ucldc@googlegroups.com" ucldc@googlegroups.com
*Subject: *[UCLDC] FW: Talend
See below from Nuxeo. Any thoughts on using a nuxeo API from talend?
-- Brian
From: Brendan Coveney [mailto:bcoveney@nuxeo.com bcoveney@nuxeo.com]
Sent: Monday, December 09, 2013 1:23 PM
To: Harlan Brown
Cc: Brian Tingle; Adrian Turner; Sherri Berger
Subject: Re: Talend
And just to be clear after speaking to Thierry, he agrees that we
"strongly discourage" the SQL injection approach.
We have a customer in France using Talend who use the REST approach and it
works really well.
Regards
Brendan
On 9 December 2013 13:08, Harlan Brown hbrown@nuxeo.com wrote:
Our CTO, Thierry Delprat (AKA Tiry) wrote some extensions for Talend
that access the Nuxeo automation API.
They're on github: https://github.com/tiry/nuxeo-talend-components
Have a look, try some test transactions, and let us know how it goes.
Thanks
--
Harlan Brown
Technical Account Manager / Technical Consultant
Nuxeo | New York
Tel. (847) 809-1813
@nuxeo | @harlanbrown
--
Brendan Coveney
General Manager Americas and Asia Pacific, Nuxeo
Phone (US): +1.408-5068630
email: bcoveney@nuxeo.com
skype: bcoveney , tweet @bcoveney
--
You received this message because you are subscribed to the Google Groups
"UCLDC project team" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ucldc+unsubscribe@googlegroups.com.
To post to this group, send email to ucldc@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying
attachments) is confidential and may contain privileged or proprietary
information that is exempt from disclosure under applicable law. If you are
not the intended recipient, or believe that you have received this message
in error, you are strictly prohibited from disclosing, copying,
distributing, or using the information contained herein. Inadvertent or
unauthorized disclosure shall not compromise or waive the confidentiality
of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately notify
us at maildisc@walkerart.org and delete this transmission from your
computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Unfortunately, there is no one "correct" way to import data into Nuxeo.
The blanket statements bandied about recently about how to and how not
to import data are incorrect and misleading. Because of the confusion
surrounding this topic, we've asked Nuxeo to create some better
documentation for importing data into Nuxeo. Here is a link to their
first draft:
http://doc.nuxeo.com/display/public/NXDOC/Importing+Data+in+Nuxeo
Please review this and send Nuxeo useful feedback.
-Richard
Chris Hoffman mailto:chris_h@berkeley.edu
Thursday, December 19, 2013 8:57 AM
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that
UCLDC conversation. However, I can say that we decided not to use
straight SQL for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting
on top of the Nuxeo ECM platform. Records are not linked together by
the kinds of patterns that normally apply. The "id" fields in each
table are not what connects them -- you have to create records in the
hierarchy table too. This pattern is used even for repeating fields
within a record type, usually that is!
b) CollectionSpace itself then does some interesting things, like
creating records in the misc table (that might be a nuxeo thing, I
don't know) and collectionspace_core. There is more here that I don't
understand.
c) To support search, Nuxeo also populates a fulltext index. This is
not done by database triggers, so the index is only updated by Nuxeo.
We're working on a way to re-index the fulltext index because we've
been using SQL to update some records.
I'd definitely be interested to hear if others have figured out how to
do migrations using straight SQL, but we've been importing XML
records. We're happy to share our techniques and code with your
contractor. It's all up in Jira, the wiki, and even github.
Regards,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Dowden, Robin mailto:robin.dowden@walkerart.org
Thursday, December 19, 2013 8:33 AM
All,
The contractor working on the Walker's data migration wants to work
directly in SQL. Seeing the note below, combined with aberrant
behavior we see when inserting records directly into tables raises
concern and gives pause. Do we know why Nuxeo strongly discourages the
SQL injection approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz <pschmitz@berkeley.edu
mailto:pschmitz@berkeley.edu>
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.org
mailto:talk@lists.collectionspace.org List"
<talk@lists.collectionspace.org mailto:talk@lists.collectionspace.org>
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is
related to another project using the Nuxeo DAMS. ANyone else using
Talend out there? Anyone tried the Nuxeo plug-in? This is new to us,
so we'll share whatever we learn.
Thanks - Patrick
Begin forwarded message:
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying
attachments) is confidential and may contain privileged or proprietary
information that is exempt from disclosure under applicable law. If
you are not the intended recipient, or believe that you have received
this message in error, you are strictly prohibited from disclosing,
copying, distributing, or using the information contained herein.
Inadvertent or unauthorized disclosure shall not compromise or waive
the confidentiality of this transmission or any applicable
attorney-client privilege.
If you have received this transmission in error, please immediately
notify us at maildisc@walkerart.org and delete this transmission from
your computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
NXQL has only SELECT support (not INSERT or UPDATE). The API supports
update and INSERT but not NXQL.
Patrick
From: Talk [mailto:talk-bounces@lists.collectionspace.org] *On
Behalf Of *Michael
Black
Sent: Thursday, December 19, 2013 9:26 AM
To: Chris Hoffman
Cc: talk@lists.collectionspace.org List
Subject: Re: [Talk] [UCLDC] FW: Talend
Hi all,
How many (if any) of the problems caused by using SQL might be avoided if
NXQL were used instead? Presumably, using NXQL would make Nuxeo "aware" of
the operations being performed, but that's just supposition on my part. If
this were the case, then perhaps at least the index-not-updating issue
might be avoided?
I imagine that if any of the processes that Chris pointed out (e.g.,
creating/updating records in misc and collectionspace_core) were handled by
CSpace services, rather than Nuxeo, the NXQL approach wouldn't be any
better than SQL for those aspects.
Of course, using any *QL approach relies on the user knowing the exact
structure of the underlying DB, which can often (but by no means always) be
predicted given the fairly consistent patterns used.
Michael
On Thu, Dec 19, 2013 at 8:57 AM, Chris Hoffman chris_h@berkeley.edu wrote:
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that UCLDC
conversation. However, I can say that we decided not to use straight SQL
for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting on
top of the Nuxeo ECM platform. Records are not linked together by the
kinds of patterns that normally apply. The "id" fields in each table are
not what connects them -- you have to create records in the hierarchy table
too. This pattern is used even for repeating fields within a record type,
usually that is!
b) CollectionSpace itself then does some interesting things, like creating
records in the misc table (that might be a nuxeo thing, I don't know) and
collectionspace_core. There is more here that I don't understand.
c) To support search, Nuxeo also populates a fulltext index. This is not
done by database triggers, so the index is only updated by Nuxeo. We're
working on a way to re-index the fulltext index because we've been using
SQL to update some records.
I'd definitely be interested to hear if others have figured out how to do
migrations using straight SQL, but we've been importing XML records. We're
happy to share our techniques and code with your contractor. It's all up
in Jira, the wiki, and even github.
Regards,
Chris
On Dec 19, 2013, at 8:33 AM, Dowden, Robin wrote:
All,
The contractor working on the Walker's data migration wants to work
directly in SQL. Seeing the note below, combined with aberrant behavior we
see when inserting records directly into tables raises concern and gives
pause. Do we know why Nuxeo strongly discourages the SQL injection
approach?
--
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
*From: *Patrick Schmitz pschmitz@berkeley.edu
*Date: *Monday, December 9, 2013 3:48 PM
*To: *"talk@lists.collectionspace.org List" talk@lists.collectionspace.org
*Subject: *[Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is
related to another project using the Nuxeo DAMS. ANyone else using Talend
out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll
share whatever we learn.
Thanks - Patrick
Begin forwarded message:
*From: *Brian Tingle Brian.Tingle@ucop.edu
*Date: *December 9, 2013 4:34:26 PM EST
*To: *"ucldc@googlegroups.com" ucldc@googlegroups.com
Subject: [UCLDC] FW: Talend
See below from Nuxeo. Any thoughts on using a nuxeo API from talend? --
Brian
From: Brendan Coveney [mailto:bcoveney@nuxeo.com bcoveney@nuxeo.com]
Sent: Monday, December 09, 2013 1:23 PM
To: Harlan Brown
Cc: Brian Tingle; Adrian Turner; Sherri Berger
Subject: Re: Talend
And just to be clear after speaking to Thierry, he agrees that we "strongly
discourage" the SQL injection approach.
We have a customer in France using Talend who use the REST approach and it
works really well.
Regards
Brendan
On 9 December 2013 13:08, Harlan Brown hbrown@nuxeo.com wrote:
Our CTO, Thierry Delprat (AKA Tiry) wrote some extensions for Talend that
access the Nuxeo automation API.
They're on github: https://github.com/tiry/nuxeo-talend-components
Have a look, try some test transactions, and let us know how it goes.
Thanks
--
Harlan Brown
Technical Account Manager / Technical Consultant
Nuxeo | New York
Tel. (847) 809-1813
@nuxeo | @harlanbrown
--
Brendan Coveney
General Manager Americas and Asia Pacific, Nuxeo
Phone (US): +1.408-5068630
email: bcoveney@nuxeo.com
skype: bcoveney , tweet @bcoveney
--
You received this message because you are subscribed to the Google Groups
"UCLDC project team" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ucldc+unsubscribe@googlegroups.com.
To post to this group, send email to ucldc@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying
attachments) is confidential and may contain privileged or proprietary
information that is exempt from disclosure under applicable law. If you are
not the intended recipient, or believe that you have received this message
in error, you are strictly prohibited from disclosing, copying,
distributing, or using the information contained herein. Inadvertent or
unauthorized disclosure shall not compromise or waive the confidentiality
of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately notify
us at maildisc@walkerart.org and delete this transmission from your
computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
This is interesting, thanks for the link, Richard.
And linking back to something Chris mentioned earlier, there is a SQL
Import Post Import Tasks table in the wiki doc that lists restarting
full-text indexing and restarting triggers. It appears Nuxeo allows this
through some mechanism...
- Jesse
On Thu, Dec 19, 2013 at 12:45 PM, Richard Millet <
richard.millet@berkeley.edu> wrote:
Unfortunately, there is no one "correct" way to import data into Nuxeo.
The blanket statements bandied about recently about how to and how not to
import data are incorrect and misleading. Because of the confusion
surrounding this topic, we've asked Nuxeo to create some better
documentation for importing data into Nuxeo. Here is a link to their first
draft:
http://doc.nuxeo.com/display/public/NXDOC/Importing+Data+in+Nuxeo
Please review this and send Nuxeo useful feedback.
-Richard
Chris Hoffman chris_h@berkeley.edu
Thursday, December 19, 2013 8:57 AM
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that UCLDC
conversation. However, I can say that we decided not to use straight SQL
for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting on
top of the Nuxeo ECM platform. Records are not linked together by the
kinds of patterns that normally apply. The "id" fields in each table are
not what connects them -- you have to create records in the hierarchy table
too. This pattern is used even for repeating fields within a record type,
usually that is!
b) CollectionSpace itself then does some interesting things, like creating
records in the misc table (that might be a nuxeo thing, I don't know) and
collectionspace_core. There is more here that I don't understand.
c) To support search, Nuxeo also populates a fulltext index. This is not
done by database triggers, so the index is only updated by Nuxeo. We're
working on a way to re-index the fulltext index because we've been using
SQL to update some records.
I'd definitely be interested to hear if others have figured out how to do
migrations using straight SQL, but we've been importing XML records. We're
happy to share our techniques and code with your contractor. It's all up
in Jira, the wiki, and even github.
Regards,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Dowden, Robin robin.dowden@walkerart.org
Thursday, December 19, 2013 8:33 AM
All,
The contractor working on the Walker's data migration wants to work
directly in SQL. Seeing the note below, combined with aberrant behavior we
see when inserting records directly into tables raises concern and gives
pause. Do we know why Nuxeo strongly discourages the SQL injection
approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz pschmitz@berkeley.edu
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.org List" talk@lists.collectionspace.org
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is
related to another project using the Nuxeo DAMS. ANyone else using Talend
out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll
share whatever we learn.
Thanks - Patrick
Begin forwarded message:
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying
attachments) is confidential and may contain privileged or proprietary
information that is exempt from disclosure under applicable law. If you are
not the intended recipient, or believe that you have received this message
in error, you are strictly prohibited from disclosing, copying,
distributing, or using the information contained herein. Inadvertent or
unauthorized disclosure shall not compromise or waive the confidentiality
of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately notify
us at maildisc@walkerart.org and delete this transmission from your
computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
In that SQL Import table, the indexing and triggers are a bit of a red
herring. The key is where it says to "Rebuild full text." Nuxeo does
provide a way to do this rebuild, but it's not currently compatible with
cspace, so I've been adapting that code into a batch job that is. See
CSPACE-6020 http://issues.collectionspace.org/browse/CSPACE-6020 and
PAHMA-964 http://issues.collectionspace.org/browse/PAHMA-964 for details.
Thanks,
Ray
On Thu, Dec 19, 2013 at 9:58 AM, Jesse Martinez mjesse@gmail.com wrote:
This is interesting, thanks for the link, Richard.
And linking back to something Chris mentioned earlier, there is a SQL
Import Post Import Tasks table in the wiki doc that lists restarting
full-text indexing and restarting triggers. It appears Nuxeo allows this
through some mechanism...
- Jesse
On Thu, Dec 19, 2013 at 12:45 PM, Richard Millet <
richard.millet@berkeley.edu> wrote:
Unfortunately, there is no one "correct" way to import data into Nuxeo.
The blanket statements bandied about recently about how to and how not to
import data are incorrect and misleading. Because of the confusion
surrounding this topic, we've asked Nuxeo to create some better
documentation for importing data into Nuxeo. Here is a link to their first
draft:
http://doc.nuxeo.com/display/public/NXDOC/Importing+Data+in+Nuxeo
Please review this and send Nuxeo useful feedback.
-Richard
Chris Hoffman chris_h@berkeley.edu
Thursday, December 19, 2013 8:57 AM
Hi Robin,
I'll ask Richard to comment on this too since he's been part of that
UCLDC conversation. However, I can say that we decided not to use straight
SQL for importing data for a number of reasons:
a) the CollectionSpace data model is very complex because it's sitting on
top of the Nuxeo ECM platform. Records are not linked together by the
kinds of patterns that normally apply. The "id" fields in each table are
not what connects them -- you have to create records in the hierarchy table
too. This pattern is used even for repeating fields within a record type,
usually that is!
b) CollectionSpace itself then does some interesting things, like
creating records in the misc table (that might be a nuxeo thing, I don't
know) and collectionspace_core. There is more here that I don't understand.
c) To support search, Nuxeo also populates a fulltext index. This is not
done by database triggers, so the index is only updated by Nuxeo. We're
working on a way to re-index the fulltext index because we've been using
SQL to update some records.
I'd definitely be interested to hear if others have figured out how to do
migrations using straight SQL, but we've been importing XML records. We're
happy to share our techniques and code with your contractor. It's all up
in Jira, the wiki, and even github.
Regards,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Dowden, Robin robin.dowden@walkerart.org
Thursday, December 19, 2013 8:33 AM
All,
The contractor working on the Walker's data migration wants to work
directly in SQL. Seeing the note below, combined with aberrant behavior we
see when inserting records directly into tables raises concern and gives
pause. Do we know why Nuxeo strongly discourages the SQL injection
approach?
Robin Dowden
Director, New Media Initiatives
Walker Art Center
1750 Hennepin Avenue
Minneapolis, MN 55403
USA
walkerart.org
t: 612.375.7541
From: Patrick Schmitz pschmitz@berkeley.edu
Date: Monday, December 9, 2013 3:48 PM
To: "talk@lists.collectionspace.org List" <talk@lists.collectionspace.org
Subject: [Talk] Fwd: [UCLDC] FW: Talend
Of possible interest to folks importing metadata into CSpace. This is
related to another project using the Nuxeo DAMS. ANyone else using Talend
out there? Anyone tried the Nuxeo plug-in? This is new to us, so we'll
share whatever we learn.
Thanks - Patrick
Begin forwarded message:
--
CONFIDENTIALITY NOTICE: This transmission (including any accompanying
attachments) is confidential and may contain privileged or proprietary
information that is exempt from disclosure under applicable law. If you are
not the intended recipient, or believe that you have received this message
in error, you are strictly prohibited from disclosing, copying,
distributing, or using the information contained herein. Inadvertent or
unauthorized disclosure shall not compromise or waive the confidentiality
of this transmission or any applicable attorney-client privilege.
If you have received this transmission in error, please immediately
notify us at maildisc@walkerart.org and delete this transmission from
your computer.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
