AR
Aron Roberts
Wed, Feb 27, 2013 4:35 PM
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
- The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
1. There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
2) It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
> On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
> <Christopher.Pott@smk.dk> wrote:
>> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
>>
>> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>>
>> Cheers,
>> Chris
>>
>>
>> -----Oprindelig meddelelse-----
>> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
>> Sendt: 26. februar 2013 21:05
>> Til: Christopher Pott
>> Cc: talk@lists.collectionspace.org
>> Emne: Re: [Talk] storing media remotely
>>
>> Hi Chris,
>>
>> This summary is my quick and thus not-well-informed look, so please
>> take this with some caveats (and corrections / comments by others are
>> welcomed):
>>
>> 1. The ims-url setting appears, from reading comments on CSPACE-4014
>> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
>> exclusively relevant to constructing image URLs to be sent from
>> Application->UI layers, for use by the UI layer in generating HTML
>> image tags.
>>
>> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>>
>>> ims-url is for the media and is for the UI to get the image from the App
>>
>> Here is the URL in a sample <img src="..."> tag for a medium-sized
>> media thumbnail on a Media Handling page on qa.collectionspace.org,
>> which points to an Application layer URL to retrieve that image:
>>
>> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>>
>> Note: there is an unresolved JIRA for allowing the UI to directly
>> retrieve resources, including digital media, from the Services layer,
>> bypassing the Application layer:
>>
>> http://issues.collectionspace.org/browse/CSPACE-4248
>>
>> 2. If BlobRead.get_blob is in fact the relevant, active method in
>> the Application layer for retrieving media, its calls to retrieve
>> binary "blobs," including digital media, are in turn sending requests
>> to the Services layer's Blob service, at a specific URL path:
>>
>> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>>
>> private void get_blob(Storage s,UIRequest q,String
>> csid,String derivative) throws UIException {
>> try {
>> JSONObject out =
>> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
>> ...
>>
>> This implies that it's the Services' Blob service that independently
>> determines the source(s) from which digital media might be obtained:
>> whether via local filesystem storage (Nuxeo-mediated or otherwise),
>> Nuxeo-based or external DAMs, or some other persistence.
>>
>> The App layer's ims-url setting appears to have no involvement with
>> that; the value of that setting, at present, simply needs to be set to
>> the hostname or IP address of the host on which the App layer is
>> running, since the UI will always (at least until CSPACE-4248 is
>> resolved) be contacting the App layer to retrieve digital media.
>>
>> Aron
>>
>> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
>> <Christopher.Pott@smk.dk> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I'm having trouble finding information on the <ims-url> tag in
>>> "local-settings.xml". Can anybody tell me which kind of remote services are
>>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>>> storage)
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Chris
>>>
>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>
>>
>> _______________________________________________
>> Talk mailing list
>> Talk@lists.collectionspace.org
>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
AR
Aron Roberts
Wed, Feb 27, 2013 4:51 PM
Finally, one more note on a somewhat related new feature in
CollectionSpace 3.2.1: Richard Millet has added a PublicItem service,
which allows selected resources in a CollectionSpace system to be
"published" - made available publicly, without requiring
authentication for access.
This would potentially allow selected resources, such as
report-generated comma-separated value (CSV) and tab-separated value
(TSV) files in one CollectionSpace system to be shared with other
museums' systems, and with any external systems or tools as well.
This 'publishing' capability is, AIUI, enabled by default for the
Reports service, but also could be enabled, for instance, for Media
Handling. (Here's how it is enabled for Reports:
https://github.com/collectionspace/services/blob/master/services/report/service/src/main/java/org/collectionspace/services/report/ReportResource.java#L155)
The initial impetus for creating the PublicItem service was to
facilitate integration with the Berkeley Mapper service
(http://code.google.com/p/berkeleymapper/), which accepts CSV
datafiles, but is also a general purpose way to expose selected
resources, including digital images and other media, for access by
external systems.
Aron
On Wed, Feb 27, 2013 at 8:35 AM, Aron Roberts
aron@socrates.berkeley.edu wrote:
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
- The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Finally, one more note on a somewhat related new feature in
CollectionSpace 3.2.1: Richard Millet has added a PublicItem service,
which allows selected resources in a CollectionSpace system to be
"published" - made available publicly, without requiring
authentication for access.
This would potentially allow selected resources, such as
report-generated comma-separated value (CSV) and tab-separated value
(TSV) files in one CollectionSpace system to be shared with other
museums' systems, and with any external systems or tools as well.
This 'publishing' capability is, AIUI, enabled by default for the
Reports service, but also could be enabled, for instance, for Media
Handling. (Here's how it is enabled for Reports:
https://github.com/collectionspace/services/blob/master/services/report/service/src/main/java/org/collectionspace/services/report/ReportResource.java#L155)
The initial impetus for creating the PublicItem service was to
facilitate integration with the Berkeley Mapper service
(http://code.google.com/p/berkeleymapper/), which accepts CSV
datafiles, but is also a general purpose way to expose selected
resources, including digital images and other media, for access by
external systems.
Aron
On Wed, Feb 27, 2013 at 8:35 AM, Aron Roberts
<aron@socrates.berkeley.edu> wrote:
> Thanks, Chris! The findings you've shared in your second paragraph
> below are terrific and, I'm sure, helpful for many of us in
> understanding CollectionSpace's current behavior around digital media
> / binary files.
>
> Just one more note: Patrick suggested that I clarify that, while
> CollectionSpace currently doesn't have any interfaces to DAMs or other
> external sources of digital media:
>
> 1. There's a lightweight way of interacting with DAMs and other
> external storage already built into in CollectionSpace: the "File
> Information -> Link to External Media" field in Media Handling
> records. (In underlying schema, this is the media_common:externalUrl
> field.)
>
> If you enter an HTTP URL in that field for an external media file
> that's publicly-accessible (does not require authentication),
> CollectionSpace will a) store that URL in the record and offer a "Go
> To" button for one-click access to the the original, and b) create and
> display locally-stored derivatives.
>
> A sample image for trying this out, by pasting this URL into the "Link
> to External Media" field:
> http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
>
> 2) It's possible to build connectors from CollectionSpace's Services
> layer to specific DAMs and other types of external storage, and it's
> our expectation that CollectionSpace will be extended with one or more
> of these, perhaps as contributions.
>
> Aron
>
>> On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
>> <Christopher.Pott@smk.dk> wrote:
>>> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
>>>
>>> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
>>>
>>> Cheers,
>>> Chris
>>>
>>>
>>> -----Oprindelig meddelelse-----
>>> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
>>> Sendt: 26. februar 2013 21:05
>>> Til: Christopher Pott
>>> Cc: talk@lists.collectionspace.org
>>> Emne: Re: [Talk] storing media remotely
>>>
>>> Hi Chris,
>>>
>>> This summary is my quick and thus not-well-informed look, so please
>>> take this with some caveats (and corrections / comments by others are
>>> welcomed):
>>>
>>> 1. The ims-url setting appears, from reading comments on CSPACE-4014
>>> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
>>> exclusively relevant to constructing image URLs to be sent from
>>> Application->UI layers, for use by the UI layer in generating HTML
>>> image tags.
>>>
>>> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>>>
>>>> ims-url is for the media and is for the UI to get the image from the App
>>>
>>> Here is the URL in a sample <img src="..."> tag for a medium-sized
>>> media thumbnail on a Media Handling page on qa.collectionspace.org,
>>> which points to an Application layer URL to retrieve that image:
>>>
>>> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>>>
>>> Note: there is an unresolved JIRA for allowing the UI to directly
>>> retrieve resources, including digital media, from the Services layer,
>>> bypassing the Application layer:
>>>
>>> http://issues.collectionspace.org/browse/CSPACE-4248
>>>
>>> 2. If BlobRead.get_blob is in fact the relevant, active method in
>>> the Application layer for retrieving media, its calls to retrieve
>>> binary "blobs," including digital media, are in turn sending requests
>>> to the Services layer's Blob service, at a specific URL path:
>>>
>>> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>>>
>>> private void get_blob(Storage s,UIRequest q,String
>>> csid,String derivative) throws UIException {
>>> try {
>>> JSONObject out =
>>> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
>>> ...
>>>
>>> This implies that it's the Services' Blob service that independently
>>> determines the source(s) from which digital media might be obtained:
>>> whether via local filesystem storage (Nuxeo-mediated or otherwise),
>>> Nuxeo-based or external DAMs, or some other persistence.
>>>
>>> The App layer's ims-url setting appears to have no involvement with
>>> that; the value of that setting, at present, simply needs to be set to
>>> the hostname or IP address of the host on which the App layer is
>>> running, since the UI will always (at least until CSPACE-4248 is
>>> resolved) be contacting the App layer to retrieve digital media.
>>>
>>> Aron
>>>
>>> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
>>> <Christopher.Pott@smk.dk> wrote:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I'm having trouble finding information on the <ims-url> tag in
>>>> "local-settings.xml". Can anybody tell me which kind of remote services are
>>>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>>>> storage)
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>> _______________________________________________
>>>> Talk mailing list
>>>> Talk@lists.collectionspace.org
>>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>>
>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
PS
Patrick Schmitz
Wed, Feb 27, 2013 9:00 PM
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
Aron Roberts aron@socrates.berkeley.edu wrote:
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
Thanks Aron for this great information, I guess we'd have to add to
the Blob Service to deal with a new media storage interface. I was
thinking we could simply configure a remote instance of Nuxeo dedicated
to binaries but I can see now that this isn't what the <ims-url> tag is
for.
In trying to understand the current media storage, I found that
images saved in Collectionspace are stored on the local file system
along a path which is built around the computed digest of the binary
(starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For
anyone else who didn't know this, I found the following page really
useful in understanding how Nuxeo references these paths, including
paths to the derivative versions of the files
http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so
take this with some caveats (and corrections / comments by others
welcomed):
- The ims-url setting appears, from reading comments on
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on
ims-url is for the media and is for the UI to get the image from
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement
that; the value of that setting, at present, simply needs to be set
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
Aron Roberts <aron@socrates.berkeley.edu> wrote:
>Thanks, Chris! The findings you've shared in your second paragraph
>below are terrific and, I'm sure, helpful for many of us in
>understanding CollectionSpace's current behavior around digital media
>/ binary files.
>
>Just one more note: Patrick suggested that I clarify that, while
>CollectionSpace currently doesn't have any interfaces to DAMs or other
>external sources of digital media:
>
>1. There's a lightweight way of interacting with DAMs and other
>external storage already built into in CollectionSpace: the "File
>Information -> Link to External Media" field in Media Handling
>records. (In underlying schema, this is the media_common:externalUrl
>field.)
>
>If you enter an HTTP URL in that field for an external media file
>that's publicly-accessible (does not require authentication),
>CollectionSpace will a) store that URL in the record and offer a "Go
>To" button for one-click access to the the original, and b) create and
>display locally-stored derivatives.
>
>A sample image for trying this out, by pasting this URL into the "Link
>to External Media" field:
>http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
>
>2) It's possible to build connectors from CollectionSpace's Services
>layer to specific DAMs and other types of external storage, and it's
>our expectation that CollectionSpace will be extended with one or more
>of these, perhaps as contributions.
>
>Aron
>
>> On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
>> <Christopher.Pott@smk.dk> wrote:
>>> Thanks Aron for this great information, I guess we'd have to add to
>the Blob Service to deal with a new media storage interface. I was
>thinking we could simply configure a remote instance of Nuxeo dedicated
>to binaries but I can see now that this isn't what the <ims-url> tag is
>for.
>>>
>>> In trying to understand the current media storage, I found that
>images saved in Collectionspace are stored on the local file system
>along a path which is built around the computed digest of the binary
>(starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For
>anyone else who didn't know this, I found the following page really
>useful in understanding how Nuxeo references these paths, including
>paths to the derivative versions of the files
>http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
>>>
>>> Cheers,
>>> Chris
>>>
>>>
>>> -----Oprindelig meddelelse-----
>>> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne
>af Aron Roberts
>>> Sendt: 26. februar 2013 21:05
>>> Til: Christopher Pott
>>> Cc: talk@lists.collectionspace.org
>>> Emne: Re: [Talk] storing media remotely
>>>
>>> Hi Chris,
>>>
>>> This summary is my quick and thus not-well-informed look, so
>please
>>> take this with some caveats (and corrections / comments by others
>are
>>> welcomed):
>>>
>>> 1. The ims-url setting appears, from reading comments on
>CSPACE-4014
>>> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
>>> exclusively relevant to constructing image URLs to be sent from
>>> Application->UI layers, for use by the UI layer in generating HTML
>>> image tags.
>>>
>>> This is borne out by Chris's comment on the Talk list on
>2011-05-27:
>>>
>>>> ims-url is for the media and is for the UI to get the image from
>the App
>>>
>>> Here is the URL in a sample <img src="..."> tag for a medium-sized
>>> media thumbnail on a Media Handling page on qa.collectionspace.org,
>>> which points to an Application layer URL to retrieve that image:
>>>
>>>
>http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>>>
>>> Note: there is an unresolved JIRA for allowing the UI to directly
>>> retrieve resources, including digital media, from the Services
>layer,
>>> bypassing the Application layer:
>>>
>>> http://issues.collectionspace.org/browse/CSPACE-4248
>>>
>>> 2. If BlobRead.get_blob is in fact the relevant, active method in
>>> the Application layer for retrieving media, its calls to retrieve
>>> binary "blobs," including digital media, are in turn sending
>requests
>>> to the Services layer's Blob service, at a specific URL path:
>>>
>>> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>>>
>>> private void get_blob(Storage s,UIRequest q,String
>>> csid,String derivative) throws UIException {
>>> try {
>>> JSONObject out =
>>> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
>>> ...
>>>
>>> This implies that it's the Services' Blob service that
>independently
>>> determines the source(s) from which digital media might be obtained:
>>> whether via local filesystem storage (Nuxeo-mediated or otherwise),
>>> Nuxeo-based or external DAMs, or some other persistence.
>>>
>>> The App layer's ims-url setting appears to have no involvement
>with
>>> that; the value of that setting, at present, simply needs to be set
>to
>>> the hostname or IP address of the host on which the App layer is
>>> running, since the UI will always (at least until CSPACE-4248 is
>>> resolved) be contacting the App layer to retrieve digital media.
>>>
>>> Aron
>>>
>>> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
>>> <Christopher.Pott@smk.dk> wrote:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I'm having trouble finding information on the <ims-url> tag in
>>>> "local-settings.xml". Can anybody tell me which kind of remote
>services are
>>>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other
>media
>>>> storage)
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>> _______________________________________________
>>>> Talk mailing list
>>>> Talk@lists.collectionspace.org
>>>>
>http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>>
>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>>
>http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>
>_______________________________________________
>Talk mailing list
>Talk@lists.collectionspace.org
>http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
CP
Christopher Pott
Thu, Feb 28, 2013 1:55 PM
Aron, if I understand your comments correctly then when using the 'externalUrl' field, only smaller derivatives are created, and the original is not stored locally but fetched each time from the url (hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media items, that if the externalUrl contains a file path (in our case a mounted network directory) then a copy of the original image (same size) is also copied to the local file system. Is it only resources identified by HTTP URLs which don't get duplicated?
Chris
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 27. februar 2013 22:01
Til: Aron Roberts; Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] Fwd: storing media remotely
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
Aron Roberts aron@socrates.berkeley.edu wrote:
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a)
store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
On Wed, Feb 27, 2013 at 5:20 AM, Christopher
Pott
Christopher.Pott@smk.dk wrote:
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
- The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment
on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.orghttp://qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App
layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Aron, if I understand your comments correctly then when using the 'externalUrl' field, only smaller derivatives are created, and the original is not stored locally but fetched each time from the url (hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media items, that if the externalUrl contains a file path (in our case a mounted network directory) then a copy of the original image (same size) is also copied to the local file system. Is it only resources identified by HTTP URLs which don't get duplicated?
Chris
________________________________
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 27. februar 2013 22:01
Til: Aron Roberts; Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] Fwd: storing media remotely
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
Aron Roberts <aron@socrates.berkeley.edu> wrote:
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
1. There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a)
store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
2) It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
On Wed, Feb 27, 2013 at 5:20 AM, Christopher
Pott
<Christopher.Pott@smk.dk> wrote:
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment
on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org<http://qa.collectionspace.org>,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
>From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App
layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
<Christopher.Pott@smk.dk> wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
________________________________
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
________________________________
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
________________________________
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
RM
Richard Millet
Thu, Feb 28, 2013 6:51 PM
Chris,
The Media and Blob Services both look for a query parameter name
'blobPurgeOrig' on 'create' and 'update' requests. If it is set to
'true' then the Services do not archive the original. The Application
layer currently always passes 'blobPurgeOrig=true' for all "external
URL" uploads (both http-based and file-based URL forms) initiated
through the UI.
If you're hitting the Services directly without setting the
'blobPurgeOrig' query param (or setting it to false) then the Services
will archive the original.
-Richard
p.s. I tested using a file-based form of a URL in the UI's "External
URL" field to upload a image. The file-based URL pointed to an image on
the local file system. The upload worked fine. Unfortunately, after
the upload, I could not use the link button to navigate to the image.
We'd have to make some changes to the UI, App, and Services to make this
work both reliably and securely.
Christopher Pott mailto:Christopher.Pott@smk.dk
Thursday, February 28, 2013 5:55 AM
Aron, if I understand your comments correctly then when using the
'externalUrl' field, only smaller derivatives are created, and the
original is not stored locally but fetched each time from the url
(hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media
items, that if the externalUrl contains a file path (in our case a
mounted network directory) then a copy of the original image (same
size) is also copied to the local file system. Is it only resources
identified by HTTP URLs which don't get duplicated?
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Patrick Schmitz mailto:pschmitz@berkeley.edu
Wednesday, February 27, 2013 1:00 PM
You can also import media from an externalURL (rather than a file
upload), so this is another alternative.
Patrick
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Aron Roberts mailto:aron@socrates.berkeley.edu
Wednesday, February 27, 2013 8:35 AM
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample<img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Christopher Pott mailto:Christopher.Pott@smk.dk
Wednesday, February 27, 2013 5:20 AM
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample<img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Aron Roberts mailto:aron@socrates.berkeley.edu
Tuesday, February 26, 2013 12:05 PM
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample<img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Chris,
The Media and Blob Services both look for a query parameter name
'blobPurgeOrig' on 'create' and 'update' requests. If it is set to
'true' then the Services do *not* archive the original. The Application
layer currently always passes 'blobPurgeOrig=true' for all "external
URL" uploads (both http-based and file-based URL forms) initiated
through the UI.
If you're hitting the Services directly without setting the
'blobPurgeOrig' query param (or setting it to false) then the Services
will archive the original.
-Richard
p.s. I tested using a file-based form of a URL in the UI's "External
URL" field to upload a image. The file-based URL pointed to an image on
the local file system. The upload worked fine. Unfortunately, after
the upload, I could not use the link button to navigate to the image.
We'd have to make some changes to the UI, App, and Services to make this
work both reliably and securely.
> Christopher Pott <mailto:Christopher.Pott@smk.dk>
> Thursday, February 28, 2013 5:55 AM
>
> Aron, if I understand your comments correctly then when using the
> 'externalUrl' field, only smaller derivatives are created, and the
> original is not stored locally but fetched each time from the url
> (hence "lightweight").
>
> I'm fairly sure that when we use the Media Service API to create media
> items, that if the externalUrl contains a file path (in our case a
> mounted network directory) then a copy of the original image (same
> size) is also copied to the local file system. Is it only resources
> identified by HTTP URLs which don't get duplicated?
>
> Chris
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Patrick Schmitz <mailto:pschmitz@berkeley.edu>
> Wednesday, February 27, 2013 1:00 PM
> You can also import media from an externalURL (rather than a file
> upload), so this is another alternative.
>
> Patrick
>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Aron Roberts <mailto:aron@socrates.berkeley.edu>
> Wednesday, February 27, 2013 8:35 AM
> Thanks, Chris! The findings you've shared in your second paragraph
> below are terrific and, I'm sure, helpful for many of us in
> understanding CollectionSpace's current behavior around digital media
> / binary files.
>
> Just one more note: Patrick suggested that I clarify that, while
> CollectionSpace currently doesn't have any interfaces to DAMs or other
> external sources of digital media:
>
> 1. There's a lightweight way of interacting with DAMs and other
> external storage already built into in CollectionSpace: the "File
> Information -> Link to External Media" field in Media Handling
> records. (In underlying schema, this is the media_common:externalUrl
> field.)
>
> If you enter an HTTP URL in that field for an external media file
> that's publicly-accessible (does not require authentication),
> CollectionSpace will a) store that URL in the record and offer a "Go
> To" button for one-click access to the the original, and b) create and
> display locally-stored derivatives.
>
> A sample image for trying this out, by pasting this URL into the "Link
> to External Media" field:
> http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
>
> 2) It's possible to build connectors from CollectionSpace's Services
> layer to specific DAMs and other types of external storage, and it's
> our expectation that CollectionSpace will be extended with one or more
> of these, perhaps as contributions.
>
> Aron
>
>> On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
>> <Christopher.Pott@smk.dk> wrote:
>>> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
>>>
>>> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
>>> Cheers,
>>> Chris
>>>
>>>
>>> -----Oprindelig meddelelse-----
>>> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
>>> Sendt: 26. februar 2013 21:05
>>> Til: Christopher Pott
>>> Cc: talk@lists.collectionspace.org
>>> Emne: Re: [Talk] storing media remotely
>>>
>>> Hi Chris,
>>>
>>> This summary is my quick and thus not-well-informed look, so please
>>> take this with some caveats (and corrections / comments by others are
>>> welcomed):
>>>
>>> 1. The ims-url setting appears, from reading comments on CSPACE-4014
>>> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
>>> exclusively relevant to constructing image URLs to be sent from
>>> Application->UI layers, for use by the UI layer in generating HTML
>>> image tags.
>>>
>>> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>>>
>>>> ims-url is for the media and is for the UI to get the image from the App
>>> Here is the URL in a sample<img src="..."> tag for a medium-sized
>>> media thumbnail on a Media Handling page on qa.collectionspace.org,
>>> which points to an Application layer URL to retrieve that image:
>>>
>>> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>>>
>>> Note: there is an unresolved JIRA for allowing the UI to directly
>>> retrieve resources, including digital media, from the Services layer,
>>> bypassing the Application layer:
>>>
>>> http://issues.collectionspace.org/browse/CSPACE-4248
>>>
>>> 2. If BlobRead.get_blob is in fact the relevant, active method in
>>> the Application layer for retrieving media, its calls to retrieve
>>> binary "blobs," including digital media, are in turn sending requests
>>> to the Services layer's Blob service, at a specific URL path:
>>>
>>> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>>>
>>> private void get_blob(Storage s,UIRequest q,String
>>> csid,String derivative) throws UIException {
>>> try {
>>> JSONObject out =
>>> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
>>> ...
>>>
>>> This implies that it's the Services' Blob service that independently
>>> determines the source(s) from which digital media might be obtained:
>>> whether via local filesystem storage (Nuxeo-mediated or otherwise),
>>> Nuxeo-based or external DAMs, or some other persistence.
>>>
>>> The App layer's ims-url setting appears to have no involvement with
>>> that; the value of that setting, at present, simply needs to be set to
>>> the hostname or IP address of the host on which the App layer is
>>> running, since the UI will always (at least until CSPACE-4248 is
>>> resolved) be contacting the App layer to retrieve digital media.
>>>
>>> Aron
>>>
>>> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
>>> <Christopher.Pott@smk.dk> wrote:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I'm having trouble finding information on the<ims-url> tag in
>>>> "local-settings.xml". Can anybody tell me which kind of remote services are
>>>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>>>> storage)
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>> _______________________________________________
>>>> Talk mailing list
>>>> Talk@lists.collectionspace.org
>>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Christopher Pott <mailto:Christopher.Pott@smk.dk>
> Wednesday, February 27, 2013 5:20 AM
> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
>
> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
> Cheers,
> Chris
>
>
> -----Oprindelig meddelelse-----
> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
> Sendt: 26. februar 2013 21:05
> Til: Christopher Pott
> Cc: talk@lists.collectionspace.org
> Emne: Re: [Talk] storing media remotely
>
> Hi Chris,
>
> This summary is my quick and thus not-well-informed look, so please
> take this with some caveats (and corrections / comments by others are
> welcomed):
>
> 1. The ims-url setting appears, from reading comments on CSPACE-4014
> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
> exclusively relevant to constructing image URLs to be sent from
> Application->UI layers, for use by the UI layer in generating HTML
> image tags.
>
> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>
>> ims-url is for the media and is for the UI to get the image from the App
>
> Here is the URL in a sample<img src="..."> tag for a medium-sized
> media thumbnail on a Media Handling page on qa.collectionspace.org,
> which points to an Application layer URL to retrieve that image:
>
> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>
> Note: there is an unresolved JIRA for allowing the UI to directly
> retrieve resources, including digital media, from the Services layer,
> bypassing the Application layer:
>
> http://issues.collectionspace.org/browse/CSPACE-4248
>
> 2. If BlobRead.get_blob is in fact the relevant, active method in
> the Application layer for retrieving media, its calls to retrieve
> binary "blobs," including digital media, are in turn sending requests
> to the Services layer's Blob service, at a specific URL path:
>
> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>
> private void get_blob(Storage s,UIRequest q,String
> csid,String derivative) throws UIException {
> try {
> JSONObject out =
> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
> ...
>
> This implies that it's the Services' Blob service that independently
> determines the source(s) from which digital media might be obtained:
> whether via local filesystem storage (Nuxeo-mediated or otherwise),
> Nuxeo-based or external DAMs, or some other persistence.
>
> The App layer's ims-url setting appears to have no involvement with
> that; the value of that setting, at present, simply needs to be set to
> the hostname or IP address of the host on which the App layer is
> running, since the UI will always (at least until CSPACE-4248 is
> resolved) be contacting the App layer to retrieve digital media.
>
> Aron
>
> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
> <Christopher.Pott@smk.dk> wrote:
>> Hi,
>>
>>
>>
>> I'm having trouble finding information on the<ims-url> tag in
>> "local-settings.xml". Can anybody tell me which kind of remote services are
>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>> storage)
>>
>>
>>
>> Thanks,
>>
>> Chris
>>
>>
>> _______________________________________________
>> Talk mailing list
>> Talk@lists.collectionspace.org
>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Aron Roberts <mailto:aron@socrates.berkeley.edu>
> Tuesday, February 26, 2013 12:05 PM
> Hi Chris,
>
> This summary is my quick and thus not-well-informed look, so please
> take this with some caveats (and corrections / comments by others are
> welcomed):
>
> 1. The ims-url setting appears, from reading comments on CSPACE-4014
> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
> exclusively relevant to constructing image URLs to be sent from
> Application->UI layers, for use by the UI layer in generating HTML
> image tags.
>
> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>
>> ims-url is for the media and is for the UI to get the image from the App
>
> Here is the URL in a sample<img src="..."> tag for a medium-sized
> media thumbnail on a Media Handling page on qa.collectionspace.org,
> which points to an Application layer URL to retrieve that image:
>
> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>
> Note: there is an unresolved JIRA for allowing the UI to directly
> retrieve resources, including digital media, from the Services layer,
> bypassing the Application layer:
>
> http://issues.collectionspace.org/browse/CSPACE-4248
>
> 2. If BlobRead.get_blob is in fact the relevant, active method in
> the Application layer for retrieving media, its calls to retrieve
> binary "blobs," including digital media, are in turn sending requests
> to the Services layer's Blob service, at a specific URL path:
>
> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>
> private void get_blob(Storage s,UIRequest q,String
> csid,String derivative) throws UIException {
> try {
> JSONObject out =
> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
> ...
>
> This implies that it's the Services' Blob service that independently
> determines the source(s) from which digital media might be obtained:
> whether via local filesystem storage (Nuxeo-mediated or otherwise),
> Nuxeo-based or external DAMs, or some other persistence.
>
> The App layer's ims-url setting appears to have no involvement with
> that; the value of that setting, at present, simply needs to be set to
> the hostname or IP address of the host on which the App layer is
> running, since the UI will always (at least until CSPACE-4248 is
> resolved) be contacting the App layer to retrieve digital media.
>
> Aron
>
> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
> <Christopher.Pott@smk.dk> wrote:
>> Hi,
>>
>>
>>
>> I'm having trouble finding information on the<ims-url> tag in
>> "local-settings.xml". Can anybody tell me which kind of remote services are
>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>> storage)
>>
>>
>>
>> Thanks,
>>
>> Chris
>>
>>
>> _______________________________________________
>> Talk mailing list
>> Talk@lists.collectionspace.org
>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
RM
Richard Millet
Thu, Feb 28, 2013 9:09 PM
I just took a look at the code, the value of the 'blobPurgeOrig' query
parameter is not of boolean type -i.e., not 'true' or 'false'.
Instead, if the query parameter is set (with any value) then the
original image is not archived. If it is not set, then the original
is archived.
Richard Millet mailto:richard.millet@berkeley.edu
Thursday, February 28, 2013 10:51 AM
Chris,
The Media and Blob Services both look for a query parameter name
'blobPurgeOrig' on 'create' and 'update' requests. If it is set to
'true' then the Services do not archive the original. The
Application layer currently always passes 'blobPurgeOrig=true' for all
"external URL" uploads (both http-based and file-based URL forms)
initiated through the UI.
If you're hitting the Services directly without setting the
'blobPurgeOrig' query param (or setting it to false) then the Services
will archive the original.
-Richard
p.s. I tested using a file-based form of a URL in the UI's "External
URL" field to upload a image. The file-based URL pointed to an image
on the local file system. The upload worked fine. Unfortunately,
after the upload, I could not use the link button to navigate to the
image. We'd have to make some changes to the UI, App, and Services to
make this work both reliably and securely.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Christopher Pott mailto:Christopher.Pott@smk.dk
Thursday, February 28, 2013 5:55 AM
Aron, if I understand your comments correctly then when using the
'externalUrl' field, only smaller derivatives are created, and the
original is not stored locally but fetched each time from the url
(hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media
items, that if the externalUrl contains a file path (in our case a
mounted network directory) then a copy of the original image (same
size) is also copied to the local file system. Is it only resources
identified by HTTP URLs which don't get duplicated?
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Patrick Schmitz mailto:pschmitz@berkeley.edu
Wednesday, February 27, 2013 1:00 PM
You can also import media from an externalURL (rather than a file
upload), so this is another alternative.
Patrick
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Aron Roberts mailto:aron@socrates.berkeley.edu
Wednesday, February 27, 2013 8:35 AM
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample<img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Christopher Pott mailto:Christopher.Pott@smk.dk
Wednesday, February 27, 2013 5:20 AM
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample<img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dk wrote:
I just took a look at the code, the value of the 'blobPurgeOrig' query
parameter is *not* of boolean type -i.e., not 'true' or 'false'.
Instead, if the query parameter is set (with any value) then the
original image is not archived. If it is *not* set, then the original
is archived.
> Richard Millet <mailto:richard.millet@berkeley.edu>
> Thursday, February 28, 2013 10:51 AM
> Chris,
>
> The Media and Blob Services both look for a query parameter name
> 'blobPurgeOrig' on 'create' and 'update' requests. If it is set to
> 'true' then the Services do *not* archive the original. The
> Application layer currently always passes 'blobPurgeOrig=true' for all
> "external URL" uploads (both http-based and file-based URL forms)
> initiated through the UI.
>
> If you're hitting the Services directly without setting the
> 'blobPurgeOrig' query param (or setting it to false) then the Services
> will archive the original.
>
> -Richard
>
> p.s. I tested using a file-based form of a URL in the UI's "External
> URL" field to upload a image. The file-based URL pointed to an image
> on the local file system. The upload worked fine. Unfortunately,
> after the upload, I could not use the link button to navigate to the
> image. We'd have to make some changes to the UI, App, and Services to
> make this work both reliably and securely.
>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Christopher Pott <mailto:Christopher.Pott@smk.dk>
> Thursday, February 28, 2013 5:55 AM
>
> Aron, if I understand your comments correctly then when using the
> 'externalUrl' field, only smaller derivatives are created, and the
> original is not stored locally but fetched each time from the url
> (hence "lightweight").
>
> I'm fairly sure that when we use the Media Service API to create media
> items, that if the externalUrl contains a file path (in our case a
> mounted network directory) then a copy of the original image (same
> size) is also copied to the local file system. Is it only resources
> identified by HTTP URLs which don't get duplicated?
>
> Chris
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Patrick Schmitz <mailto:pschmitz@berkeley.edu>
> Wednesday, February 27, 2013 1:00 PM
> You can also import media from an externalURL (rather than a file
> upload), so this is another alternative.
>
> Patrick
>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Aron Roberts <mailto:aron@socrates.berkeley.edu>
> Wednesday, February 27, 2013 8:35 AM
> Thanks, Chris! The findings you've shared in your second paragraph
> below are terrific and, I'm sure, helpful for many of us in
> understanding CollectionSpace's current behavior around digital media
> / binary files.
>
> Just one more note: Patrick suggested that I clarify that, while
> CollectionSpace currently doesn't have any interfaces to DAMs or other
> external sources of digital media:
>
> 1. There's a lightweight way of interacting with DAMs and other
> external storage already built into in CollectionSpace: the "File
> Information -> Link to External Media" field in Media Handling
> records. (In underlying schema, this is the media_common:externalUrl
> field.)
>
> If you enter an HTTP URL in that field for an external media file
> that's publicly-accessible (does not require authentication),
> CollectionSpace will a) store that URL in the record and offer a "Go
> To" button for one-click access to the the original, and b) create and
> display locally-stored derivatives.
>
> A sample image for trying this out, by pasting this URL into the "Link
> to External Media" field:
> http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
>
> 2) It's possible to build connectors from CollectionSpace's Services
> layer to specific DAMs and other types of external storage, and it's
> our expectation that CollectionSpace will be extended with one or more
> of these, perhaps as contributions.
>
> Aron
>
>> On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
>> <Christopher.Pott@smk.dk> wrote:
>>> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
>>>
>>> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
>>> Cheers,
>>> Chris
>>>
>>>
>>> -----Oprindelig meddelelse-----
>>> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
>>> Sendt: 26. februar 2013 21:05
>>> Til: Christopher Pott
>>> Cc: talk@lists.collectionspace.org
>>> Emne: Re: [Talk] storing media remotely
>>>
>>> Hi Chris,
>>>
>>> This summary is my quick and thus not-well-informed look, so please
>>> take this with some caveats (and corrections / comments by others are
>>> welcomed):
>>>
>>> 1. The ims-url setting appears, from reading comments on CSPACE-4014
>>> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
>>> exclusively relevant to constructing image URLs to be sent from
>>> Application->UI layers, for use by the UI layer in generating HTML
>>> image tags.
>>>
>>> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>>>
>>>> ims-url is for the media and is for the UI to get the image from the App
>>> Here is the URL in a sample<img src="..."> tag for a medium-sized
>>> media thumbnail on a Media Handling page on qa.collectionspace.org,
>>> which points to an Application layer URL to retrieve that image:
>>>
>>> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>>>
>>> Note: there is an unresolved JIRA for allowing the UI to directly
>>> retrieve resources, including digital media, from the Services layer,
>>> bypassing the Application layer:
>>>
>>> http://issues.collectionspace.org/browse/CSPACE-4248
>>>
>>> 2. If BlobRead.get_blob is in fact the relevant, active method in
>>> the Application layer for retrieving media, its calls to retrieve
>>> binary "blobs," including digital media, are in turn sending requests
>>> to the Services layer's Blob service, at a specific URL path:
>>>
>>> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>>>
>>> private void get_blob(Storage s,UIRequest q,String
>>> csid,String derivative) throws UIException {
>>> try {
>>> JSONObject out =
>>> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
>>> ...
>>>
>>> This implies that it's the Services' Blob service that independently
>>> determines the source(s) from which digital media might be obtained:
>>> whether via local filesystem storage (Nuxeo-mediated or otherwise),
>>> Nuxeo-based or external DAMs, or some other persistence.
>>>
>>> The App layer's ims-url setting appears to have no involvement with
>>> that; the value of that setting, at present, simply needs to be set to
>>> the hostname or IP address of the host on which the App layer is
>>> running, since the UI will always (at least until CSPACE-4248 is
>>> resolved) be contacting the App layer to retrieve digital media.
>>>
>>> Aron
>>>
>>> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
>>> <Christopher.Pott@smk.dk> wrote:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I'm having trouble finding information on the<ims-url> tag in
>>>> "local-settings.xml". Can anybody tell me which kind of remote services are
>>>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>>>> storage)
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>> _______________________________________________
>>>> Talk mailing list
>>>> Talk@lists.collectionspace.org
>>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>>>
>>> _______________________________________________
>>> Talk mailing list
>>> Talk@lists.collectionspace.org
>>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
> Christopher Pott <mailto:Christopher.Pott@smk.dk>
> Wednesday, February 27, 2013 5:20 AM
> Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the<ims-url> tag is for.
>
> In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
>
> Cheers,
> Chris
>
>
> -----Oprindelig meddelelse-----
> Fra: aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
> Sendt: 26. februar 2013 21:05
> Til: Christopher Pott
> Cc: talk@lists.collectionspace.org
> Emne: Re: [Talk] storing media remotely
>
> Hi Chris,
>
> This summary is my quick and thus not-well-informed look, so please
> take this with some caveats (and corrections / comments by others are
> welcomed):
>
> 1. The ims-url setting appears, from reading comments on CSPACE-4014
> and CSPACE-4051 (particularly Chris Martin's), to be primarily or
> exclusively relevant to constructing image URLs to be sent from
> Application->UI layers, for use by the UI layer in generating HTML
> image tags.
>
> This is borne out by Chris's comment on the Talk list on 2011-05-27:
>
>> ims-url is for the media and is for the UI to get the image from the App
>
> Here is the URL in a sample<img src="..."> tag for a medium-sized
> media thumbnail on a Media Handling page on qa.collectionspace.org,
> which points to an Application layer URL to retrieve that image:
>
> http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
>
> Note: there is an unresolved JIRA for allowing the UI to directly
> retrieve resources, including digital media, from the Services layer,
> bypassing the Application layer:
>
> http://issues.collectionspace.org/browse/CSPACE-4248
>
> 2. If BlobRead.get_blob is in fact the relevant, active method in
> the Application layer for retrieving media, its calls to retrieve
> binary "blobs," including digital media, are in turn sending requests
> to the Services layer's Blob service, at a specific URL path:
>
> From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
>
> private void get_blob(Storage s,UIRequest q,String
> csid,String derivative) throws UIException {
> try {
> JSONObject out =
> s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
> ...
>
> This implies that it's the Services' Blob service that independently
> determines the source(s) from which digital media might be obtained:
> whether via local filesystem storage (Nuxeo-mediated or otherwise),
> Nuxeo-based or external DAMs, or some other persistence.
>
> The App layer's ims-url setting appears to have no involvement with
> that; the value of that setting, at present, simply needs to be set to
> the hostname or IP address of the host on which the App layer is
> running, since the UI will always (at least until CSPACE-4248 is
> resolved) be contacting the App layer to retrieve digital media.
>
> Aron
>
> On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
> <Christopher.Pott@smk.dk> wrote:
>> Hi,
>>
>>
>>
>> I'm having trouble finding information on the<ims-url> tag in
>> "local-settings.xml". Can anybody tell me which kind of remote services are
>> supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
>> storage)
>>
>>
>>
>> Thanks,
>>
>> Chris
>>
>>
>> _______________________________________________
>> Talk mailing list
>> Talk@lists.collectionspace.org
>> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
>>
>
> _______________________________________________
> Talk mailing list
> Talk@lists.collectionspace.org
> http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
CP
Christopher Pott
Mon, Mar 4, 2013 10:10 AM
Thanks Richard for this information. It's encouraging that the Media functionality is designed to be flexible enough to match our requirements. These requirements are still rather fluid as we're between DAM solutions right now and so have to support the current functionality (mounted file system) as well as the future (possibly HTTP url but not certain).
Cheers,
Chris
Fra: Richard Millet [mailto:richard.millet@berkeley.edu]
Sendt: 28. februar 2013 22:10
Til: Richard Millet
Cc: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] Fwd: storing media remotely
I just took a look at the code, the value of the 'blobPurgeOrig' query parameter is not of boolean type -i.e., not 'true' or 'false'. Instead, if the query parameter is set (with any value) then the original image is not archived. If it is not set, then the original is archived.
[cid:image001.jpg@01CE18C7.748C8DD0]
Richard Milletmailto:richard.millet@berkeley.edu
Thursday, February 28, 2013 10:51 AM
Chris,
The Media and Blob Services both look for a query parameter name 'blobPurgeOrig' on 'create' and 'update' requests. If it is set to 'true' then the Services do not archive the original. The Application layer currently always passes 'blobPurgeOrig=true' for all "external URL" uploads (both http-based and file-based URL forms) initiated through the UI.
If you're hitting the Services directly without setting the 'blobPurgeOrig' query param (or setting it to false) then the Services will archive the original.
-Richard
p.s. I tested using a file-based form of a URL in the UI's "External URL" field to upload a image. The file-based URL pointed to an image on the local file system. The upload worked fine. Unfortunately, after the upload, I could not use the link button to navigate to the image. We'd have to make some changes to the UI, App, and Services to make this work both reliably and securely.
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Christopher Pottmailto:Christopher.Pott@smk.dk
Thursday, February 28, 2013 5:55 AM
Aron, if I understand your comments correctly then when using the 'externalUrl' field, only smaller derivatives are created, and the original is not stored locally but fetched each time from the url (hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media items, that if the externalUrl contains a file path (in our case a mounted network directory) then a copy of the original image (same size) is also copied to the local file system. Is it only resources identified by HTTP URLs which don't get duplicated?
Chris
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Patrick Schmitzmailto:pschmitz@berkeley.edu
Wednesday, February 27, 2013 1:00 PM
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Aron Robertsmailto:aron@socrates.berkeley.edu
Wednesday, February 27, 2013 8:35 AM
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
- There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-foto_1.jpg/397px-Augusto_De_Luca-_foto_1.jpg
- It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
Christopher.Pott@smk.dkmailto:Christopher.Pott@smk.dk wrote:
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.commailto:aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.orgmailto:talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
- The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dkmailto:Christopher.Pott@smk.dk wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Christopher Pottmailto:Christopher.Pott@smk.dk
Wednesday, February 27, 2013 5:20 AM
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.commailto:aronroberts@gmail.com [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.orgmailto:talk@lists.collectionspace.org
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
- The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
- If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
Christopher.Pott@smk.dkmailto:Christopher.Pott@smk.dk wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.orgmailto:Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Thanks Richard for this information. It's encouraging that the Media functionality is designed to be flexible enough to match our requirements. These requirements are still rather fluid as we're between DAM solutions right now and so have to support the current functionality (mounted file system) as well as the future (possibly HTTP url but not certain).
Cheers,
Chris
________________________________
Fra: Richard Millet [mailto:richard.millet@berkeley.edu]
Sendt: 28. februar 2013 22:10
Til: Richard Millet
Cc: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] Fwd: storing media remotely
I just took a look at the code, the value of the 'blobPurgeOrig' query parameter is *not* of boolean type -i.e., not 'true' or 'false'. Instead, if the query parameter is set (with any value) then the original image is not archived. If it is *not* set, then the original is archived.
[cid:image001.jpg@01CE18C7.748C8DD0]
Richard Millet<mailto:richard.millet@berkeley.edu>
Thursday, February 28, 2013 10:51 AM
Chris,
The Media and Blob Services both look for a query parameter name 'blobPurgeOrig' on 'create' and 'update' requests. If it is set to 'true' then the Services do *not* archive the original. The Application layer currently always passes 'blobPurgeOrig=true' for all "external URL" uploads (both http-based and file-based URL forms) initiated through the UI.
If you're hitting the Services directly without setting the 'blobPurgeOrig' query param (or setting it to false) then the Services will archive the original.
-Richard
p.s. I tested using a file-based form of a URL in the UI's "External URL" field to upload a image. The file-based URL pointed to an image on the local file system. The upload worked fine. Unfortunately, after the upload, I could not use the link button to navigate to the image. We'd have to make some changes to the UI, App, and Services to make this work both reliably and securely.
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Christopher Pott<mailto:Christopher.Pott@smk.dk>
Thursday, February 28, 2013 5:55 AM
Aron, if I understand your comments correctly then when using the 'externalUrl' field, only smaller derivatives are created, and the original is not stored locally but fetched each time from the url (hence "lightweight").
I'm fairly sure that when we use the Media Service API to create media items, that if the externalUrl contains a file path (in our case a mounted network directory) then a copy of the original image (same size) is also copied to the local file system. Is it only resources identified by HTTP URLs which don't get duplicated?
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Patrick Schmitz<mailto:pschmitz@berkeley.edu>
Wednesday, February 27, 2013 1:00 PM
You can also import media from an externalURL (rather than a file upload), so this is another alternative.
Patrick
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Aron Roberts<mailto:aron@socrates.berkeley.edu>
Wednesday, February 27, 2013 8:35 AM
Thanks, Chris! The findings you've shared in your second paragraph
below are terrific and, I'm sure, helpful for many of us in
understanding CollectionSpace's current behavior around digital media
/ binary files.
Just one more note: Patrick suggested that I clarify that, while
CollectionSpace currently doesn't have any interfaces to DAMs or other
external sources of digital media:
1. There's a lightweight way of interacting with DAMs and other
external storage already built into in CollectionSpace: the "File
Information -> Link to External Media" field in Media Handling
records. (In underlying schema, this is the media_common:externalUrl
field.)
If you enter an HTTP URL in that field for an external media file
that's publicly-accessible (does not require authentication),
CollectionSpace will a) store that URL in the record and offer a "Go
To" button for one-click access to the the original, and b) create and
display locally-stored derivatives.
A sample image for trying this out, by pasting this URL into the "Link
to External Media" field:
http://upload.wikimedia.org/wikipedia/commons/thumb/7/72/Augusto_De_Luca_-_foto_1.jpg/397px-Augusto_De_Luca_-_foto_1.jpg
2) It's possible to build connectors from CollectionSpace's Services
layer to specific DAMs and other types of external storage, and it's
our expectation that CollectionSpace will be extended with one or more
of these, perhaps as contributions.
Aron
On Wed, Feb 27, 2013 at 5:20 AM, Christopher Pott
<Christopher.Pott@smk.dk><mailto:Christopher.Pott@smk.dk> wrote:
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com<mailto:aronroberts@gmail.com> [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org<mailto:talk@lists.collectionspace.org>
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
>From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
<Christopher.Pott@smk.dk><mailto:Christopher.Pott@smk.dk> wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
[cid:image001.jpg@01CE18C7.748C8DD0]
Christopher Pott<mailto:Christopher.Pott@smk.dk>
Wednesday, February 27, 2013 5:20 AM
Thanks Aron for this great information, I guess we'd have to add to the Blob Service to deal with a new media storage interface. I was thinking we could simply configure a remote instance of Nuxeo dedicated to binaries but I can see now that this isn't what the <ims-url> tag is for.
In trying to understand the current media storage, I found that images saved in Collectionspace are stored on the local file system along a path which is built around the computed digest of the binary (starting at "$CSPACE_JEESERVER_HOME/nuxeo-server/binaries/data/"). For anyone else who didn't know this, I found the following page really useful in understanding how Nuxeo references these paths, including paths to the derivative versions of the files http://answers.nuxeo.com/questions/2761/thumbnail-paths .
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: aronroberts@gmail.com<mailto:aronroberts@gmail.com> [mailto:aronroberts@gmail.com] På vegne af Aron Roberts
Sendt: 26. februar 2013 21:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org<mailto:talk@lists.collectionspace.org>
Emne: Re: [Talk] storing media remotely
Hi Chris,
This summary is my quick and thus not-well-informed look, so please
take this with some caveats (and corrections / comments by others are
welcomed):
1. The ims-url setting appears, from reading comments on CSPACE-4014
and CSPACE-4051 (particularly Chris Martin's), to be primarily or
exclusively relevant to constructing image URLs to be sent from
Application->UI layers, for use by the UI layer in generating HTML
image tags.
This is borne out by Chris's comment on the Talk list on 2011-05-27:
ims-url is for the media and is for the UI to get the image from the App
Here is the URL in a sample <img src="..."> tag for a medium-sized
media thumbnail on a Media Handling page on qa.collectionspace.org,
which points to an Application layer URL to retrieve that image:
http://qa.collectionspace.org:8180/collectionspace/tenant/core/download/923fd34c-c5d2-4d0f-bafb/Medium
Note: there is an unresolved JIRA for allowing the UI to directly
retrieve resources, including digital media, from the Services layer,
bypassing the Application layer:
http://issues.collectionspace.org/browse/CSPACE-4248
2. If BlobRead.get_blob is in fact the relevant, active method in
the Application layer for retrieving media, its calls to retrieve
binary "blobs," including digital media, are in turn sending requests
to the Services layer's Blob service, at a specific URL path:
>From org.collectionspace.chain.csp.webui.mediablob.BlobRead:
private void get_blob(Storage s,UIRequest q,String
csid,String derivative) throws UIException {
try {
JSONObject out =
s.retrieveJSON("/blobs/"+csid+"/"+derivative,null);
...
This implies that it's the Services' Blob service that independently
determines the source(s) from which digital media might be obtained:
whether via local filesystem storage (Nuxeo-mediated or otherwise),
Nuxeo-based or external DAMs, or some other persistence.
The App layer's ims-url setting appears to have no involvement with
that; the value of that setting, at present, simply needs to be set to
the hostname or IP address of the host on which the App layer is
running, since the UI will always (at least until CSPACE-4248 is
resolved) be contacting the App layer to retrieve digital media.
Aron
On Tue, Feb 26, 2013 at 10:14 AM, Christopher Pott
<Christopher.Pott@smk.dk><mailto:Christopher.Pott@smk.dk> wrote:
Hi,
I'm having trouble finding information on the <ims-url> tag in
"local-settings.xml". Can anybody tell me which kind of remote services are
supported? (eg. full Collectionspace, Nuxeo only, Nuxeo DAM, other media
storage)
Thanks,
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org<mailto:Talk@lists.collectionspace.org>
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
