talk@lists.collectionspace.org

WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org

View all threads

what's the best way to represent structured dates?

PS
Patrick Schmitz
Fri, Aug 26, 2011 6:16 PM

Chris - I added you as a CC on CSPACE-4347
http://issues.collectionspace.org/browse/CSPACE-4347 . If it is a big deal
for you, you might want to vote for it (or other bugs that you really want
fixed). Megan and I are looking at 1.12 priorities right now.

Patrick


From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of Michael T.
Black
Sent: Friday, August 26, 2011 7:16 AM
To: Christopher Pott
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent structured dates?

Hi Chris,

I made the mistake of assuming a Jira had already been filed, since no one
seemed surprised to hear that I was having this problem.  I didn't find a
Jira for this issue, so I created CSPACE-4347.

Michael

On Aug 26, 2011, at 2:13 AM, Christopher Pott wrote:

Hi Michael,

Thanks for sharing this, it's helpful to understand how you've approached
this. What is the problem with the repeating groups? Is it an issue with
your local configuration or is there an oustanding JIRA?

My only fear with replacing core dates, (and some other core fields) is that
other functionality (eg. Search, summary lists) may rely upon certain core
fields being present. But possibly all this can be simply configured and
this is an unjust concern?

Cheers,

Chris


Fra: Michael T. Black [mailto:mtblack@berkeley.edu]
Sendt: 25. august 2011 20:03
Til: Chris Hoffman
Cc: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?

Hi Chris and Chris,

                I've changed a fair number of the dates in the PAHMA

deployment to structured dates.  And by "changed" I actually mean
replaced.  It was about as straightforward as far as changes to schemas
go.  I put the new structured dates in a domain schema (anthropology) and
left the original calendar dates unchanged, as they're in the common schema.
I'll be happy to share any details not obvious from the following.  Pretty
much all I had to do to change these calendar dates in
collectionobjects_common.xsd:

                       <xs:element name="inscriptionContentDate"

type="xs:string"/>
<xs:element name="inscriptionDescriptionDate"
type="xs:string"/>
<xs:element name="contentDate" type="xs:string"/>

            Was create these structured dates in

collectionobjects_anthropology.xsd:

                       <xs:element name="inscriptionContentDateGroup"

type="inscriptionContentDateGroup"/>
<xs:element
name="inscriptionDescriptionDateGroup"
type="inscriptionDescriptionDateGroup"/>
<xs:element name="contentDateGroup"
type="contentDateGroup"/>

                       <xs:complexType

name="inscriptionContentDateGroup">
xs:sequence

<xs:element name="dateDisplayDate" type="xs:string"/>

<xs:element name="dateAssociation" type="xs:string"/>

<xs:element name="dateEarliestSingleYear" type="xs:string"/>

<xs:element name="dateEarliestSingleMonth" type="xs:string"/>

<xs:element name="dateEarliestSingleDay" type="xs:string"/>

<xs:element name="dateEarliestSingleEra" type="xs:string"/>

<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>

<xs:element name="dateLatestYear" type="xs:string"/>

<xs:element name="dateLatestMonth" type="xs:string"/>

<xs:element name="dateLatestDay" type="xs:string"/>

<xs:element name="dateLatestEra" type="xs:string"/>

<xs:element name="dateLatestCertainty" type="xs:string"/>

<xs:element name="dateLatestQualifier" type="xs:string"/>

<xs:element name="dateLatestQualifierValue" type="xs:string"/>

<xs:element name="dateLatestQualifierUnit" type="xs:string"/>

<xs:element name="datePeriod" type="xs:string"/>

<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>

                       <xs:complexType

name="inscriptionDescriptionDateGroup">
xs:sequence

<xs:element name="dateDisplayDate" type="xs:string"/>

<xs:element name="dateAssociation" type="xs:string"/>

<xs:element name="dateEarliestSingleYear" type="xs:string"/>

<xs:element name="dateEarliestSingleMonth" type="xs:string"/>

<xs:element name="dateEarliestSingleDay" type="xs:string"/>

<xs:element name="dateEarliestSingleEra" type="xs:string"/>

<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>

<xs:element name="dateLatestYear" type="xs:string"/>

<xs:element name="dateLatestMonth" type="xs:string"/>

<xs:element name="dateLatestDay" type="xs:string"/>

<xs:element name="dateLatestEra" type="xs:string"/>

<xs:element name="dateLatestCertainty" type="xs:string"/>

<xs:element name="dateLatestQualifier" type="xs:string"/>

<xs:element name="dateLatestQualifierValue" type="xs:string"/>

<xs:element name="dateLatestQualifierUnit" type="xs:string"/>

<xs:element name="datePeriod" type="xs:string"/>

<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>

                       <xs:complexType name="contentDateGroup">
                                                    <xs:sequence>

<xs:element name="dateDisplayDate" type="xs:string"/>

<xs:element name="dateAssociation" type="xs:string"/>

<xs:element name="dateEarliestSingleYear" type="xs:string"/>

<xs:element name="dateEarliestSingleMonth" type="xs:string"/>

<xs:element name="dateEarliestSingleDay" type="xs:string"/>

<xs:element name="dateEarliestSingleEra" type="xs:string"/>

<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>

<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>

<xs:element name="dateLatestYear" type="xs:string"/>

<xs:element name="dateLatestMonth" type="xs:string"/>

<xs:element name="dateLatestDay" type="xs:string"/>

<xs:element name="dateLatestEra" type="xs:string"/>

<xs:element name="dateLatestCertainty" type="xs:string"/>

<xs:element name="dateLatestQualifier" type="xs:string"/>

<xs:element name="dateLatestQualifierValue" type="xs:string"/>

<xs:element name="dateLatestQualifierUnit" type="xs:string"/>

<xs:element name="datePeriod" type="xs:string"/>

<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>

Then I commented out the old dates in our-tenant-tenant.xml (with
mini-build, otherwise in your tenant's xxxx-tenant.xml file):

                    <field id="inscriptionContentDate" ui-type="date">

<selector>object-description-inscription-content-date</selector>
</field>

                       <field id="inscriptionDescriptionDate"

ui-type="date">

<selector>object-description-inscription-description-date</selector>
</field>

                    <field id="contentDate" ui-type="date" >

<selector>object-description-content-date</selector>
</field>

And I inserted these instead:

                       <field id="inscriptionContentDate"

section="anthropology" ui-type="groupfield/structureddate"
primarykey="inscriptionContentDate.dateDisplayDate"
ui-func="cspace.structuredDate"  >

<services-tag>inscriptionContentDateGroup</services-tag>

<selector>object-description-inscription-content-date</selector>
</field>

                       <field id="inscriptionDescriptionDate"

section="anthropology" ui-type="groupfield/structureddate"
primarykey="inscriptionDescriptionDate.dateDisplayDate"
ui-func="cspace.structuredDate"  >

<services-tag>inscriptionDescriptionDateGroup</services-tag>

<selector>object-description-inscription-description-date</selector>
</field>

                       <field id="contentDate" section="anthropology"

ui-type="groupfield/structureddate" primarykey="contentDate.dateDisplayDate"
ui-func="cspace.structuredDate"  >

<services-tag>contentDateGroup</services-tag>

<selector>object-description-content-date</selector>
</field>

NB: I didn't have to change/add anything in core-messages.properties, as the
new dates have the same ids and selectors as the old dates.

Then I rebuilt/redeployed the App and Service layers and restart both
servers and the new dates worked as expected.

The only problem I've had is that you can't [yet] place structured dates
into repeating groups (group lists) and have them function properly.

Michael

On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote:

Hi Chris,

We'll be doing the same thing to a lot of dates -- making them structured by
changing them or replacing them.  We've been told it's not hard to change
the type, but we don't have much experience with it yet.  As we learn more,
I'll ask the team to post some tips and tricks.

Thanks,

Chris

On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:

Hi Chris,

No, we've not yet changed any exisiting calendar dates to structured dates,
although we feel most "non administritive" date fields would be better
served by structured dates (though this distinction is not always
straightforward).

I'm still uncertain about this area - presumeably we can't just replace a
calendar date from the core schema but would have to create a structured
date extension field and use that instead? And then would we be able to
perform advanced search on these new dates?

Cheers,

Chris


Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?

Hi Chris,

Thanks for asking these questions.  We've been wondering the same things.
Have you changed any existing calendar date fields to structured dates or
added new structured date fields?  I think this is all quite new, and
together we might be able to come up with some best practices (related to
data migration and data entry).

I suspect that search and advanced search of structured date fields is going
to be an issue for those of us deploying CollectionSpace.  Our users will
want CSpace to be smart about structured dates, interpreting qualifiers, and
so on, but I bet that we won't see much as much as we would like in the
first version of advanced search.  I'm always happy to be proved wrong!  :-)

Chris

On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:

Hi,

I'm looking at the structured date schema and the best way to represent
dates migrated from our old system. Any information about the following
would be much appreciated..

  1. I see some different/overlapping possibilities for recording an object
    creation date. For example, to represent "1718" (display date) one could
    either store:

a) a single date, providing year only (1718)

b) an earliest date (01.01.1718) with a latest date (31.12.1718)

c) a single date (01.01.1718) with a qualifier of + 364 days

Is there any difference in the meaning of these three (and would they all be
treated equally by services such as advanced search)? Our migration data is
currently in format 'b' but the other formats are sometimes 'friendlier' for
new input.

  1. How are date "certainties" and "qualifiers" used by advanced search?

I've specified "1718", as a creation date with a certainty of "circa". Will
a search for objects created during 1717 result in a hit?

Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact
limits for this type of searching?

  1. Is the Date schema locally extendable? (We need a field to hold an
    English translation of the display text)

  2. Will Advanced search by creation date also search the display date or
    will it just use the single/earliest and latest date groups?

Thanks,

Chris


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace
.org


Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace
.org

Chris - I added you as a CC on CSPACE-4347 <http://issues.collectionspace.org/browse/CSPACE-4347> . If it is a big deal for you, you might want to vote for it (or other bugs that you really want fixed). Megan and I are looking at 1.12 priorities right now. Patrick _____ From: talk-bounces@lists.collectionspace.org [mailto:talk-bounces@lists.collectionspace.org] On Behalf Of Michael T. Black Sent: Friday, August 26, 2011 7:16 AM To: Christopher Pott Cc: talk@lists.collectionspace.org Subject: Re: [Talk] what's the best way to represent structured dates? Hi Chris, I made the mistake of assuming a Jira had already been filed, since no one seemed surprised to hear that I was having this problem. I didn't find a Jira for this issue, so I created CSPACE-4347. Michael On Aug 26, 2011, at 2:13 AM, Christopher Pott wrote: Hi Michael, Thanks for sharing this, it's helpful to understand how you've approached this. What is the problem with the repeating groups? Is it an issue with your local configuration or is there an oustanding JIRA? My only fear with replacing core dates, (and some other core fields) is that other functionality (eg. Search, summary lists) may rely upon certain core fields being present. But possibly all this can be simply configured and this is an unjust concern? Cheers, Chris _____ Fra: Michael T. Black [mailto:mtblack@berkeley.edu] Sendt: 25. august 2011 20:03 Til: Chris Hoffman Cc: Christopher Pott; talk@lists.collectionspace.org Emne: Re: [Talk] what's the best way to represent structured dates? Hi Chris and Chris, I've changed a fair number of the dates in the PAHMA deployment to structured dates. And by "changed" I actually mean *replaced.* It was about as straightforward as far as changes to schemas go. I put the new structured dates in a domain schema (anthropology) and left the original calendar dates unchanged, as they're in the common schema. I'll be happy to share any details not obvious from the following. Pretty much all I had to do to change these calendar dates in collectionobjects_common.xsd: <xs:element name="inscriptionContentDate" type="xs:string"/> <xs:element name="inscriptionDescriptionDate" type="xs:string"/> <xs:element name="contentDate" type="xs:string"/> Was create these structured dates in collectionobjects_anthropology.xsd: <xs:element name="inscriptionContentDateGroup" type="inscriptionContentDateGroup"/> <xs:element name="inscriptionDescriptionDateGroup" type="inscriptionDescriptionDateGroup"/> <xs:element name="contentDateGroup" type="contentDateGroup"/> <xs:complexType name="inscriptionContentDateGroup"> <xs:sequence> <xs:element name="dateDisplayDate" type="xs:string"/> <xs:element name="dateAssociation" type="xs:string"/> <xs:element name="dateEarliestSingleYear" type="xs:string"/> <xs:element name="dateEarliestSingleMonth" type="xs:string"/> <xs:element name="dateEarliestSingleDay" type="xs:string"/> <xs:element name="dateEarliestSingleEra" type="xs:string"/> <xs:element name="dateEarliestSingleCertainty" type="xs:string"/> <xs:element name="dateEarliestSingleQualifier" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/> <xs:element name="dateLatestYear" type="xs:string"/> <xs:element name="dateLatestMonth" type="xs:string"/> <xs:element name="dateLatestDay" type="xs:string"/> <xs:element name="dateLatestEra" type="xs:string"/> <xs:element name="dateLatestCertainty" type="xs:string"/> <xs:element name="dateLatestQualifier" type="xs:string"/> <xs:element name="dateLatestQualifierValue" type="xs:string"/> <xs:element name="dateLatestQualifierUnit" type="xs:string"/> <xs:element name="datePeriod" type="xs:string"/> <xs:element name="dateNote" type="xs:string"/> </xs:sequence> </xs:complexType> <xs:complexType name="inscriptionDescriptionDateGroup"> <xs:sequence> <xs:element name="dateDisplayDate" type="xs:string"/> <xs:element name="dateAssociation" type="xs:string"/> <xs:element name="dateEarliestSingleYear" type="xs:string"/> <xs:element name="dateEarliestSingleMonth" type="xs:string"/> <xs:element name="dateEarliestSingleDay" type="xs:string"/> <xs:element name="dateEarliestSingleEra" type="xs:string"/> <xs:element name="dateEarliestSingleCertainty" type="xs:string"/> <xs:element name="dateEarliestSingleQualifier" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/> <xs:element name="dateLatestYear" type="xs:string"/> <xs:element name="dateLatestMonth" type="xs:string"/> <xs:element name="dateLatestDay" type="xs:string"/> <xs:element name="dateLatestEra" type="xs:string"/> <xs:element name="dateLatestCertainty" type="xs:string"/> <xs:element name="dateLatestQualifier" type="xs:string"/> <xs:element name="dateLatestQualifierValue" type="xs:string"/> <xs:element name="dateLatestQualifierUnit" type="xs:string"/> <xs:element name="datePeriod" type="xs:string"/> <xs:element name="dateNote" type="xs:string"/> </xs:sequence> </xs:complexType> <xs:complexType name="contentDateGroup"> <xs:sequence> <xs:element name="dateDisplayDate" type="xs:string"/> <xs:element name="dateAssociation" type="xs:string"/> <xs:element name="dateEarliestSingleYear" type="xs:string"/> <xs:element name="dateEarliestSingleMonth" type="xs:string"/> <xs:element name="dateEarliestSingleDay" type="xs:string"/> <xs:element name="dateEarliestSingleEra" type="xs:string"/> <xs:element name="dateEarliestSingleCertainty" type="xs:string"/> <xs:element name="dateEarliestSingleQualifier" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/> <xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/> <xs:element name="dateLatestYear" type="xs:string"/> <xs:element name="dateLatestMonth" type="xs:string"/> <xs:element name="dateLatestDay" type="xs:string"/> <xs:element name="dateLatestEra" type="xs:string"/> <xs:element name="dateLatestCertainty" type="xs:string"/> <xs:element name="dateLatestQualifier" type="xs:string"/> <xs:element name="dateLatestQualifierValue" type="xs:string"/> <xs:element name="dateLatestQualifierUnit" type="xs:string"/> <xs:element name="datePeriod" type="xs:string"/> <xs:element name="dateNote" type="xs:string"/> </xs:sequence> </xs:complexType> Then I commented out the old dates in our-tenant-tenant.xml (with mini-build, otherwise in your tenant's xxxx-tenant.xml file): <field id="inscriptionContentDate" ui-type="date"> <selector>object-description-inscription-content-date</selector> </field> <field id="inscriptionDescriptionDate" ui-type="date"> <selector>object-description-inscription-description-date</selector> </field> <field id="contentDate" ui-type="date" > <selector>object-description-content-date</selector> </field> And I inserted these instead: <field id="inscriptionContentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionContentDate.dateDisplayDate" ui-func="cspace.structuredDate" > <services-tag>inscriptionContentDateGroup</services-tag> <selector>object-description-inscription-content-date</selector> </field> <field id="inscriptionDescriptionDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionDescriptionDate.dateDisplayDate" ui-func="cspace.structuredDate" > <services-tag>inscriptionDescriptionDateGroup</services-tag> <selector>object-description-inscription-description-date</selector> </field> <field id="contentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="contentDate.dateDisplayDate" ui-func="cspace.structuredDate" > <services-tag>contentDateGroup</services-tag> <selector>object-description-content-date</selector> </field> NB: I didn't have to change/add anything in core-messages.properties, as the new dates have the same ids and selectors as the old dates. Then I rebuilt/redeployed the App and Service layers and restart both servers and the new dates worked as expected. The only problem I've had is that you can't [yet] place structured dates into repeating groups (group lists) and have them function properly. Michael On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote: Hi Chris, We'll be doing the same thing to a lot of dates -- making them structured by changing them or replacing them. We've been told it's not hard to change the type, but we don't have much experience with it yet. As we learn more, I'll ask the team to post some tips and tricks. Thanks, Chris On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote: Hi Chris, No, we've not yet changed any exisiting calendar dates to structured dates, although we feel most "non administritive" date fields would be better served by structured dates (though this distinction is not always straightforward). I'm still uncertain about this area - presumeably we can't just replace a calendar date from the core schema but would have to create a structured date extension field and use that instead? And then would we be able to perform advanced search on these new dates? Cheers, Chris _____ Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu] Sendt: 23. august 2011 20:05 Til: Christopher Pott Cc: talk@lists.collectionspace.org Emne: Re: [Talk] what's the best way to represent structured dates? Hi Chris, Thanks for asking these questions. We've been wondering the same things. Have you changed any existing calendar date fields to structured dates or added new structured date fields? I think this is all quite new, and together we might be able to come up with some best practices (related to data migration and data entry). I suspect that search and advanced search of structured date fields is going to be an issue for those of us deploying CollectionSpace. Our users will want CSpace to be smart about structured dates, interpreting qualifiers, and so on, but I bet that we won't see much as much as we would like in the first version of advanced search. I'm always happy to be proved wrong! :-) Chris On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote: Hi, I'm looking at the structured date schema and the best way to represent dates migrated from our old system. Any information about the following would be much appreciated.. 1. I see some different/overlapping possibilities for recording an object creation date. For example, to represent "1718" (display date) one could either store: a) a single date, providing year only (1718) b) an earliest date (01.01.1718) with a latest date (31.12.1718) c) a single date (01.01.1718) with a qualifier of + 364 days Is there any difference in the meaning of these three (and would they all be treated equally by services such as advanced search)? Our migration data is currently in format 'b' but the other formats are sometimes 'friendlier' for new input. 2. How are date "certainties" and "qualifiers" used by advanced search? I've specified "1718", as a creation date with a certainty of "circa". Will a search for objects created during 1717 result in a hit? Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact limits for this type of searching? 3. Is the Date schema locally extendable? (We need a field to hold an English translation of the display text) 4. Will Advanced search by creation date also search the display date or will it just use the single/earliest and latest date groups? Thanks, Chris _______________________________________________ Talk mailing list Talk@lists.collectionspace.org http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace .org _______________________________________________ Talk mailing list Talk@lists.collectionspace.org http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace .org
CP
Christopher Pott
Mon, Aug 29, 2011 12:41 PM

Hi Patrick,

The semantics of our date descriptors are described in the pdf slides at the top of the requirements page @ http://wiki.collectionspace.org/display/collectionspace/Date+and+Time+Use+Cases .

I'll contact you regarding the possibilities for cooperation on this area.

Cheers,
Chris

-----Oprindelig meddelelse-----
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 26. august 2011 20:10
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: RE: [Talk] what's the best way to represent structured dates?

Hi Chris -

Perhaps we can talk about a batch process or validation handler that will
produce the dates you want. If you can get the semantic details worked
out,
then we can consider a submission to the core project. I would be happy to
work with you on how to make this work.

Patrick

-----Original Message-----
From: Christopher Pott [mailto:Christopher.Pott@smk.dk]
Sent: Friday, August 26, 2011 5:42 AM
To: Patrick Schmitz; talk@lists.collectionspace.org
Subject: SV: [Talk] what's the best way to represent structured dates?

Hi Patrick,

That's great, this information allows us to progress with the
migration work for structured date. From what I understand
the problem of extending repeating core schemas (including
date) is covered by CSPACE-3894 and the remaining Date work
by CSPACE-3943.

I appreciate that the long term aim is to allow fuzzy dates
and interpretation of qualifiers (is there a JIRA for this?),
but I'm concerned this may be beyond the timeframe of our
(SMK's) first expected delivery.

I think the first iteration can support our needs, it just
places a little more burden on our UI to generate a second
date in some cases where a qualifier would have been simpler,
but I don't see this as a problem.

Thanks,
Chris

-----Oprindelig meddelelse-----
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 26. august 2011 02:08
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: RE: [Talk] what's the best way to represent structured dates?

Hi Chris -

I believe your interpretations are correct. Note that the first
iteration will ignore fuzzy qualifiers, but we'd like to:

  1. Allow some adjustment of search based upon these qualifiers
  2. Allow local specification of what each qualifier means

I think Date should be extensible, but we will run into the

issue we

have elsewhere raised, that if the date is in a repeating section,
adding another column to each row is not currently possible

(you would

have to duplicate the whole row in your extension schema).

We've been

talking about solutions that would be more reasonable for
implementers.

Patrick

-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of
Christopher Pott
Sent: Thursday, August 25, 2011 5:14 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent

structured dates?

Thanks for the responses. Can you tell me if these

interpretations

of your responses are correct....

  • if we always provide a full (dd-MM-yyyy GG) earliest and latest
    date, these will be the dates used by services for search and
    ordering
  • if we provide less information (such as just a year in earliest
    date) services will generate an internal earliest and

latest date to

cover that period

  • Services won't be parsing display dates to generate

earliest and

latest dates

  • Services won't be examining  'qualifiers' or 'certainty'
    fields so if we rely on these then searching will fail

Is the date schema extensible? We need a way to store some
information with a date, which probably wouldn't be useful to
services but that can help us modify the UI to our requirements.

Our earliest structured date is currently 1600 BC which I thought
was quite old until I read the other posts.

Just out of interest to other implementers, some screen

shots of our

planned date UI implementation can be seen here
http://wiki.collectionspace.org/display/collectionspace/Date+a
nd+Time+Use+Cases. It will use a combination of drop down
lists and day/month/year inputs to enable generation of display
dates according to some precise rules (required for

catalogues and

signs). It should generate a translation at the same

time, as well

as the earliest and latest dates.

Cheers,
Chris

-----Oprindelig meddelelse-----
Fra: talk-bounces@lists.collectionspace.org [mailto:talk-
bounces@lists.collectionspace.org] På vegne af Patrick Schmitz
Sendt: 24. august 2011 23:47
Til: 'Michael T. Black'
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent

structured dates?

So this is what I more or less expected.

I see several alternatives:

  1. We do something special for Paleo
  2. We implement our own Calendar support that can get more

range out

of a Long storage unit.

2 would be a lot more work, and I am not thrilled with that

approach.

I am pretty sure I do not know enough about how Paleo

and friends

enter dates, interpret dates, compare them, etc., etc. If they
actually use a different model for these dates, then

perhaps 1 is a reasonable approach.

E.g., we could support a different scalar date field

that used a

different unit (like years, rather than days), and then

they could do

searches on that rather than date fields. A long range of

years into

the past (E.g., rooted at 3000 CE) could represent years

nearly back

to the beginning of our planet (about 4.2 billion years ago).
While astronomic time has a larger range, we can cross

that bridge

with a similar approach (e.g., an extension field that

represents decades).

Patrick

-----Original Message-----
From: Michael T. Black [mailto:mtblack@berkeley.edu]
Sent: Wednesday, August 24, 2011 1:59 PM
To: Patrick Schmitz
Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo
Pellegatti
Subject: Re: [Talk] what's the best way to represent

structured dates?

Hi Patrick,

Just concluded a summit of the chronometric working

group here at

the Hearst (or, as we call it, me and Paolo), and we're

happy with a

300Myr limit (BTW-we and presumably paleo, geology, etc.

would call

this BP, not BCE).  We have a handful of 50-150Myr BP objects
for which we might conceivably have dates we'd want to enter.

However,

we can't think of any cases in which we've got objects older
than 300Myr for which the = 300Myr date would have enough

importance to

make entry of these dates a requirement for us at the

moment (e.g.,

we might have a bead made of an Ordovician

brachiopod, but the

collection and production dates would be the critical

ones for us,

not the date of the material from which the item was

made).  I say

"at the moment" because it would be awfully handy to

have these

dates entered once we're at the point where we can offer

federated

searching across collections/museums.

I can tell you, though, that you'll probably

have a tough time

convincing the paleo folks that they don't need to enter

dates for

work for

Michael, for Paleo, Essig, et al. I am worried about Paleo

in particular.

Any other folks on this list with collections that go back

a long way?

How much of a limitation would the 300M yr BCE range be?

Thanks - Patrick

Behalf Of Aron

Roberts
Sent: Wednesday, August 24, 2011 11:39 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent

structured dates?

On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund
janice.l.eklund@gmail.com wrote:

I see no problem with the scalar date scenarios

listed below.

Thanks much, Patrick, for such a simple, sensible way of
creating single range start and end values, and

Jan for your

feedback that this would work for you.

I've had many long discussions with Susan and Glen about

this topic

and their frustrations with 'black box' or otherwise

unsatisfactory

fuzzy date algorithms.  Using our current model /

schema for

structured dates, with its separate Year/Month/Day fields

for range

start/single date and end date - for which they both get

considerable

credit - and the clear logic you outline below, date range

search can

be transparent and easy to understand.

Note that entry of Year/Month/Day values versus

uncertainty in the

structure date popup is an either/or, as I understand

it.  If you

want the transparent, easy-to-understand behavior, you

enter a year,

and maybe also a month, or month and day, for the

start/single date,

and
(optionally) for the end date.  If you enter

uncertainty-related

values, when this is implemented down the road, perhaps a

pluggable

fuzzy date handler would be invoked, a la Dick's

suggestion?

Aron

On 8/24/11, Patrick Schmitz pschmitz@berkeley.edu wrote:

Maybe I am missing something, but is there a problem with

the following:

Single date, expressed as "1910". We model this (in
earliest and latest scalar dates) as esd: "Jan 1

1910" and

lsd: "Dec 31 1910"

Date range expressed as earliest:"1910",

latest:"1915". We

model this

as
esd: "Jan 1 1910" and lsd: "Dec 31 1915"

Date range expressed as earliest:"Jan 1910", latest:"Aug

1910". We

model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910"

The esd and lsd scalar dates are used for search and

ordering, not display.

Patrick

On Behalf

Of Janice

Eklund
Sent: Wednesday, August 24, 2011 10:17 AM
To: Susan Stone
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent

structured dates?

I second Susan's concern about false precision.

At least

90% of our

dates are year only single dates or date ranges. If you
are considering fuzzy logic down the road, look at the

existing code

carefully.  The fuzzy date machine, at least as

implemented at UCB

for the HAVRC, does not handle ancient dates properly.

Jan

On 8/24/11, Susan Stone

Patrick,

There was quite a bit of discussion of this in our

Wednesday afternoon

meetings at Berkeley that I'm sure Aron will recall. I

don't think

date fields go back far enough in time, so I'd

recommend

using integers or floating point numbers and providing
the math

in search.

It is also important to make sure that false

precision isn't

introduced. If something is dated 1900, the system

shouldn't enter

that as January 1, 1900 unless the uncertainty

values can

be used to

differentiate and the display date can be simply 1900.

Susan

On 08/24/2011 09:16 AM, Patrick Schmitz wrote:

Really? I am a little surprised that in all

the design

discussions for structured dates this did not come up.
Anyway, I will say that we do not have the

time to build

some system

that allows for a lot of custom behavior, or elaborate

heuristics

here (i.e., trying to divine what is intended from a

set of text

fields). I would strongly suggest that we either

  1. require entry of some actual date fields

upon which

the application

 can reason,
 or
  1. we define a very simple set of rules for building
    such

date values

 from some of the existing text fields. By very

simple,

I mean that

 we only consider fields like year, month, day, and

that the rules

 produce two scalar dates (early and late), which

either define a

 range, or define a point (if they are

equal). AIUI,

a

structured

 date /cannot/ model the idea of "some time/any

time after (or

 before) a date".

One further question is the semantic of which date (if

any) will be

used for ordering.
If and when advanced search supports date

ranges (which

also implies

support for /before/ and /after/ logic in search), we
can

handle the

interval logic associated with the structured

dates, but

only if we

have a pair of scalar date values.
Down the road, I do think it would be interesting to

implement some

fuzzier logic on date search, to better

support "circa"

logic and the

uncertainty values. However, I suspect that it will

take longer

to arrive at consensus on these semantics, and would

defer that

discussion for now.
Thanks - Patrick



 *From:* Chris Hoffman
 *Sent:* Wednesday, August 24, 2011 9:01 AM
 *To:* Patrick Schmitz
 *Cc:* 'Christopher Pott'; 'Aron Roberts';

talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to
represent

structured

dates?

 Hi Patrick, I think it's fair to say there's no

consensus because

 there's been no discussion :-) It's possible that

each

customer will

 want to use structured dates differently,

which will

make it hard to

 add any intelligence or even predictability to

things

like search.

 However, I also think that if a

significant number

of deploying
institutions can come to some consensus on best

ways to use

and
search structured dates, then many

institutions will

choose to adopt

 the same best practices.

 Thanks,
 Chris

 On Aug 24, 2011, at 8:55 AM, Patrick

Schmitz wrote:

 Aron was out yesterday, and Rick is out for a few

days. They have

 done a lot of the structured dates work to date.
 The last I heard, Aron was hoping to get

consensus

from the users

 about how we map all the text fields to

actual date

fields that we

 can reason on (for search, ordering,

etc). To date,

there seems to

 be no such consensus, which will preclude search,

ordering, etc.

 on structured dates. I would think that

this would

make structured

 dates quite a bit less useful...
 Perhaps (I hope) my understanding is out

of date,

and

we just need

 to implement the logic. Perhaps Aron can jump in

this

AM, when he

 gets in.
 Patrick


     *From:*

*On Behalf Of

     *Chris Hoffman
     *Sent:* Tuesday, August 23, 2011 11:05 AM
     *To:* Christopher Pott
     *Cc:* talk@lists.collectionspace.org
     <mailto:talk@lists.collectionspace.org>
     *Subject:* Re: [Talk] what's the best way to

represent

     structured dates?

     Hi Chris,

     Thanks for asking these questions. We've been

wondering the

     same things. Have you changed any existing

calendar date

     fields to structured dates or added new

structured date

     fields? I think this is all quite new, and

together we might

     be able to come up with some best practices

(related to data

     migration and data entry).

     I suspect that search and advanced search of

structured date

     fields is going to be an issue for those of

us deploying

     CollectionSpace. Our users will want

CSpace to

be

smart about

     structured dates, interpreting

qualifiers, and

so

on, but I

     bet that we won't see much as much as we

would like in

the
first version of advanced search. I'm always

happy to be

     proved wrong! :-)

     Chris

     On Aug 23, 2011, at 8:48 AM, Christopher

Pott wrote:

     Hi,

     I'm looking at the structured date

schema and

the best way to

     represent dates migrated from our

old system.

Any information

     about the following would be much

appreciated..

     1. I see some different/overlapping

possibilities for

     recording an object creation date. For

example,

to represent

     "1718" (display date) one could

either store:

     a) a single date, providing year only (1718)

     b) an earliest date (01.01.1718)

with a latest

date
(31.12.1718)

     c) a single date (01.01.1718) with a

qualifier of + 364

days

     Is there any difference in the meaning of

these

three (and

     would they all be treated equally by

services such as

     advanced search)? Our migration data is

currently in format

     'b' but the other formats are sometimes

'friendlier' for new

     input.

     2. How are date "certainties" and

"qualifiers" used by

     advanced search?

     I've specified "1718", as a

creation date with

a

certainty of

     "circa". Will a search for objects created

during 1717 result

     in a hit?

     Or is it necessary to use

qualifiers (eg. 1718

+/- 2 years)

     to define exact limits for this type of

searching?

     3. Is the Date schema locally

extendable? (We

need a field to

     hold an English translation of the display

text)

     4. Will Advanced search by creation date

also search

the
display date or will it just use the

single/earliest

and
latest date groups?

     Thanks,

     Chris

     Talk mailing list
     Talk@lists.collectionspace.org
     <mailto:Talk@lists.collectionspace.org>

ec

tionspace.org


Talk mailing list
Talk@lists.collectionspace.org

i

onspace.org


Talk mailing list
Talk@lists.collectionspace.org

i

o

nspace.org


Talk mailing list
Talk@lists.collectionspace.org

c
ollectionspace.org


Talk mailing list
Talk@lists.collectionspace.org

Hi Patrick, The semantics of our date descriptors are described in the pdf slides at the top of the requirements page @ http://wiki.collectionspace.org/display/collectionspace/Date+and+Time+Use+Cases . I'll contact you regarding the possibilities for cooperation on this area. Cheers, Chris > -----Oprindelig meddelelse----- > Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu] > Sendt: 26. august 2011 20:10 > Til: Christopher Pott; talk@lists.collectionspace.org > Emne: RE: [Talk] what's the best way to represent structured dates? > > Hi Chris - > > Perhaps we can talk about a batch process or validation handler that will > produce the dates you want. If you can get the semantic details worked > out, > then we can consider a submission to the core project. I would be happy to > work with you on how to make this work. > > Patrick > > > -----Original Message----- > > From: Christopher Pott [mailto:Christopher.Pott@smk.dk] > > Sent: Friday, August 26, 2011 5:42 AM > > To: Patrick Schmitz; talk@lists.collectionspace.org > > Subject: SV: [Talk] what's the best way to represent structured dates? > > > > Hi Patrick, > > > > That's great, this information allows us to progress with the > > migration work for structured date. From what I understand > > the problem of extending repeating core schemas (including > > date) is covered by CSPACE-3894 and the remaining Date work > > by CSPACE-3943. > > > > I appreciate that the long term aim is to allow fuzzy dates > > and interpretation of qualifiers (is there a JIRA for this?), > > but I'm concerned this may be beyond the timeframe of our > > (SMK's) first expected delivery. > > > > I think the first iteration can support our needs, it just > > places a little more burden on our UI to generate a second > > date in some cases where a qualifier would have been simpler, > > but I don't see this as a problem. > > > > Thanks, > > Chris > > > > > -----Oprindelig meddelelse----- > > > Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu] > > > Sendt: 26. august 2011 02:08 > > > Til: Christopher Pott; talk@lists.collectionspace.org > > > Emne: RE: [Talk] what's the best way to represent structured dates? > > > > > > Hi Chris - > > > > > > I believe your interpretations are correct. Note that the first > > > iteration will ignore fuzzy qualifiers, but we'd like to: > > > > > > 1) Allow some adjustment of search based upon these qualifiers > > > 2) Allow local specification of what each qualifier means > > > > > > I think Date should be extensible, but we will run into the > > issue we > > > have elsewhere raised, that if the date is in a repeating section, > > > adding another column to each row is not currently possible > > (you would > > > have to duplicate the whole row in your extension schema). > > We've been > > > talking about solutions that would be more reasonable for > > > implementers. > > > > > > Patrick > > > > > > > -----Original Message----- > > > > From: talk-bounces@lists.collectionspace.org > > > > [mailto:talk-bounces@lists.collectionspace.org] On Behalf Of > > > > Christopher Pott > > > > Sent: Thursday, August 25, 2011 5:14 AM > > > > To: talk@lists.collectionspace.org > > > > Subject: Re: [Talk] what's the best way to represent > > structured dates? > > > > > > > > Thanks for the responses. Can you tell me if these > > interpretations > > > > of your responses are correct.... > > > > > > > > * if we always provide a full (dd-MM-yyyy GG) earliest and latest > > > > date, these will be the dates used by services for search and > > > > ordering > > > > * if we provide less information (such as just a year in earliest > > > > date) services will generate an internal earliest and > > latest date to > > > > cover that period > > > > * Services won't be parsing display dates to generate > > earliest and > > > > latest dates > > > > * Services won't be examining 'qualifiers' or 'certainty' > > > > fields so if we rely on these then searching will fail > > > > > > > > Is the date schema extensible? We need a way to store some > > > > information with a date, which probably wouldn't be useful to > > > > services but that can help us modify the UI to our requirements. > > > > > > > > Our earliest structured date is currently 1600 BC which I thought > > > > was quite old until I read the other posts. > > > > > > > > Just out of interest to other implementers, some screen > > shots of our > > > > planned date UI implementation can be seen here > > > > http://wiki.collectionspace.org/display/collectionspace/Date+a > > > > nd+Time+Use+Cases. It will use a combination of drop down > > > > lists and day/month/year inputs to enable generation of display > > > > dates according to some precise rules (required for > > catalogues and > > > > signs). It should generate a translation at the same > > time, as well > > > > as the earliest and latest dates. > > > > > > > > Cheers, > > > > Chris > > > > > > > > > -----Oprindelig meddelelse----- > > > > > Fra: talk-bounces@lists.collectionspace.org [mailto:talk- > > > > > bounces@lists.collectionspace.org] På vegne af Patrick Schmitz > > > > > Sendt: 24. august 2011 23:47 > > > > > Til: 'Michael T. Black' > > > > > Cc: talk@lists.collectionspace.org > > > > > Emne: Re: [Talk] what's the best way to represent > > structured dates? > > > > > > > > > > So this is what I more or less expected. > > > > > > > > > > I see several alternatives: > > > > > > > > > > 1) We do something special for Paleo > > > > > 2) We implement our own Calendar support that can get more > > > > range out > > > > > of a Long storage unit. > > > > > > > > > > 2 would be a lot more work, and I am not thrilled with that > > > > approach. > > > > > > > > > > I am pretty sure I do not know enough about how Paleo > > and friends > > > > > enter dates, interpret dates, compare them, etc., etc. If they > > > > > actually use a different model for these dates, then > > > > perhaps 1 is a reasonable approach. > > > > > E.g., we could support a different scalar date field > > that used a > > > > > different unit (like years, rather than days), and then > > > > they could do > > > > > searches on that rather than date fields. A long range of > > > > years into > > > > > the past (E.g., rooted at 3000 CE) could represent years > > > > nearly back > > > > > to the beginning of our planet (about 4.2 billion years ago). > > > > > While astronomic time has a larger range, we can cross > > that bridge > > > > > with a similar approach (e.g., an extension field that > > represents decades). > > > > > > > > > > Patrick > > > > > > > > > > > -----Original Message----- > > > > > > From: Michael T. Black [mailto:mtblack@berkeley.edu] > > > > > > Sent: Wednesday, August 24, 2011 1:59 PM > > > > > > To: Patrick Schmitz > > > > > > Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo > > > > > > Pellegatti > > > > > > Subject: Re: [Talk] what's the best way to represent > > > > structured dates? > > > > > > > > > > > > Hi Patrick, > > > > > > > > > > > > Just concluded a summit of the chronometric working > > > > group here at > > > > > > the Hearst (or, as we call it, me and Paolo), and we're > > > > happy with a > > > > > > 300Myr limit (BTW-we and presumably paleo, geology, etc. > > > > would call > > > > > > this BP, not BCE). We have a handful of 50-150Myr BP objects > > > > > > for which we might conceivably have dates we'd want to enter. > > > > However, > > > > > > we can't think of any cases in which we've got objects older > > > > > > than 300Myr for which the = 300Myr date would have enough > > > > importance to > > > > > > make entry of these dates a requirement for us at the > > > > moment (e.g., > > > > > > we might have a bead made of an Ordovician > > brachiopod, but the > > > > > > collection and production dates would be the critical > > > > ones for us, > > > > > > not the date of the material from which the item was > > > > made). I say > > > > > > "at the moment" because it would be awfully handy to > > have these > > > > > > dates entered once we're at the point where we can offer > > > > federated > > > > > > searching across collections/museums. > > > > > > > > > > > > I can tell you, though, that you'll probably > > have a tough time > > > > > > convincing the paleo folks that they don't need to enter > > > > dates for > > > > > > their Carboniferous and earlier collections: > > > > > > > > > > > > http://www.ucmp.berkeley.edu/science/inverts/carboniferous.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/pennsylvanian.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/mississippian.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/devonian.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/silurian.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/ordovician.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/cambrian.php > > > > > > http://www.ucmp.berkeley.edu/science/inverts/precambrian.php > > > > > > > > > > > > Michael > > > > > > > > > > > > On Aug 24, 2011, at 12:00 PM, Patrick Schmitz wrote: > > > > > > > > > > > > > I think there is lots of potential for fuzzy date > > > > > > interpretation, and > > > > > > > we can pursue this as a second step. > > > > > > > > > > > > > > Next, we need to find out if the 300M yr BCE range will > > > > work for > > > > > > > Michael, for Paleo, Essig, et al. I am worried about Paleo > > > > > > in particular. > > > > > > > > > > > > > > Any other folks on this list with collections that go back > > > > > > a long way? > > > > > > > How much of a limitation would the 300M yr BCE range be? > > > > > > > > > > > > > > Thanks - Patrick > > > > > > > > > > > > > >> -----Original Message----- > > > > > > >> From: talk-bounces@lists.collectionspace.org > > > > > > >> [mailto:talk-bounces@lists.collectionspace.org] On > > > > Behalf Of Aron > > > > > > >> Roberts > > > > > > >> Sent: Wednesday, August 24, 2011 11:39 AM > > > > > > >> To: talk@lists.collectionspace.org > > > > > > >> Subject: Re: [Talk] what's the best way to represent > > > > > > structured dates? > > > > > > >> > > > > > > >> On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund > > > > > > >> <janice.l.eklund@gmail.com> wrote: > > > > > > >>> I see no problem with the scalar date scenarios > > listed below. > > > > > > >> > > > > > > >> Thanks much, Patrick, for such a simple, sensible way of > > > > > > >> creating single range start and end values, and > > Jan for your > > > > > > >> feedback that this would work for you. > > > > > > >> > > > > > > >> I've had many long discussions with Susan and Glen about > > > > > > this topic > > > > > > >> and their frustrations with 'black box' or otherwise > > > > > > unsatisfactory > > > > > > >> fuzzy date algorithms. Using our current model / > > schema for > > > > > > >> structured dates, with its separate Year/Month/Day fields > > > > > > for range > > > > > > >> start/single date and end date - for which they both get > > > > > > considerable > > > > > > >> credit - and the clear logic you outline below, date range > > > > > > search can > > > > > > >> be transparent and easy to understand. > > > > > > >> > > > > > > >> Note that entry of Year/Month/Day values versus > > > > > > uncertainty in the > > > > > > >> structure date popup is an either/or, as I understand > > > > it. If you > > > > > > >> want the transparent, easy-to-understand behavior, you > > > > > > enter a year, > > > > > > >> and maybe also a month, or month and day, for the > > > > > > start/single date, > > > > > > >> and > > > > > > >> (optionally) for the end date. If you enter > > > > uncertainty-related > > > > > > >> values, when this is implemented down the road, perhaps a > > > > > > pluggable > > > > > > >> fuzzy date handler would be invoked, a la Dick's > > suggestion? > > > > > > >> > > > > > > >> Aron > > > > > > >> > > > > > > >>> On 8/24/11, Patrick Schmitz <pschmitz@berkeley.edu> wrote: > > > > > > >>>> Maybe I am missing something, but is there a problem with > > > > > > >> the following: > > > > > > >>>> > > > > > > >>>> Single date, expressed as "1910". We model this (in > > > > > > >>>> earliest and latest scalar dates) as esd: "Jan 1 > > 1910" and > > > > lsd: "Dec 31 1910" > > > > > > >>>> > > > > > > >>>> Date range expressed as earliest:"1910", > > latest:"1915". We > > > > > > >> model this > > > > > > >>>> as > > > > > > >>>> esd: "Jan 1 1910" and lsd: "Dec 31 1915" > > > > > > >>>> > > > > > > >>>> Date range expressed as earliest:"Jan 1910", latest:"Aug > > > > > > 1910". We > > > > > > >>>> model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910" > > > > > > >>>> > > > > > > >>>> The esd and lsd scalar dates are used for search and > > > > > > >> ordering, not display. > > > > > > >>>> > > > > > > >>>> Patrick > > > > > > >>>> > > > > > > >>>>> -----Original Message----- > > > > > > >>>>> From: talk-bounces@lists.collectionspace.org > > > > > > >>>>> [mailto:talk-bounces@lists.collectionspace.org] > > On Behalf > > > > > > >> Of Janice > > > > > > >>>>> Eklund > > > > > > >>>>> Sent: Wednesday, August 24, 2011 10:17 AM > > > > > > >>>>> To: Susan Stone > > > > > > >>>>> Cc: talk@lists.collectionspace.org > > > > > > >>>>> Subject: Re: [Talk] what's the best way to represent > > > > > > >> structured dates? > > > > > > >>>>> > > > > > > >>>>> I second Susan's concern about false precision. > > At least > > > > > > >> 90% of our > > > > > > >>>>> dates are year only single dates or date ranges. If you > > > > > > >>>>> are considering fuzzy logic down the road, look at the > > > > > > existing code > > > > > > >>>>> carefully. The fuzzy date machine, at least as > > > > > > >> implemented at UCB > > > > > > >>>>> for the HAVRC, does not handle ancient dates properly. > > > > > > >>>>> > > > > > > >>>>> Jan > > > > > > >>>>> > > > > > > >>>>> On 8/24/11, Susan Stone > > > > <sstone@socrates.berkeley.edu> wrote: > > > > > > >>>>>> Patrick, > > > > > > >>>>>> > > > > > > >>>>>> There was quite a bit of discussion of this in our > > > > > > >>>>> Wednesday afternoon > > > > > > >>>>>> meetings at Berkeley that I'm sure Aron will recall. I > > > > > > >> don't think > > > > > > >>>>>> date fields go back far enough in time, so I'd > > recommend > > > > > > >>>>>> using integers or floating point numbers and providing > > > > > > >>>>>> the math > > > > > > >>>>> in search. > > > > > > >>>>>> It is also important to make sure that false > > > > precision isn't > > > > > > >>>>>> introduced. If something is dated 1900, the system > > > > > > >> shouldn't enter > > > > > > >>>>>> that as January 1, 1900 unless the uncertainty > > values can > > > > > > >>>>> be used to > > > > > > >>>>>> differentiate and the display date can be simply 1900. > > > > > > >>>>>> > > > > > > >>>>>> Susan > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> On 08/24/2011 09:16 AM, Patrick Schmitz wrote: > > > > > > >>>>>>> Really? I am a little surprised that in all > > the design > > > > > > >>>>>>> discussions for structured dates this did not come up. > > > > > > >>>>>>> Anyway, I will say that we do not have the > > time to build > > > > > > >>>>> some system > > > > > > >>>>>>> that allows for a lot of custom behavior, or elaborate > > > > > > >> heuristics > > > > > > >>>>>>> here (i.e., trying to divine what is intended from a > > > > > > >> set of text > > > > > > >>>>>>> fields). I would strongly suggest that we either > > > > > > >>>>>>> > > > > > > >>>>>>> 1. > > > > > > >>>>>>> require entry of some actual date fields > > upon which > > > > > > >>>>> the application > > > > > > >>>>>>> can reason, > > > > > > >>>>>>> or > > > > > > >>>>>>> 2. > > > > > > >>>>>>> we define a very simple set of rules for building > > > > > > >>>>>>> such > > > > > > >>>>> date values > > > > > > >>>>>>> from some of the existing text fields. By very > > > > > > >>>>>>> simple, > > > > > > >>>>> I mean that > > > > > > >>>>>>> we only consider fields like year, month, day, and > > > > > > >>>>> that the rules > > > > > > >>>>>>> produce two scalar dates (early and late), which > > > > > > >>>>> either define a > > > > > > >>>>>>> range, or define a point (if they are > > equal). AIUI, > > > > > > >>>>>>> a > > > > > > >>>>> structured > > > > > > >>>>>>> date /cannot/ model the idea of "some time/any > > > > > > >> time after (or > > > > > > >>>>>>> before) a date". > > > > > > >>>>>>> > > > > > > >>>>>>> One further question is the semantic of which date (if > > > > > > >>>>> any) will be > > > > > > >>>>>>> used for ordering. > > > > > > >>>>>>> If and when advanced search supports date > > ranges (which > > > > > > >>>>> also implies > > > > > > >>>>>>> support for /before/ and /after/ logic in search), we > > > > > > >>>>>>> can > > > > > > >>>>> handle the > > > > > > >>>>>>> interval logic associated with the structured > > dates, but > > > > > > >>>>> only if we > > > > > > >>>>>>> have a pair of scalar date values. > > > > > > >>>>>>> Down the road, I do think it would be interesting to > > > > > > >>>>> implement some > > > > > > >>>>>>> fuzzier logic on date search, to better > > support "circa" > > > > > > >>>>> logic and the > > > > > > >>>>>>> uncertainty values. However, I suspect that it will > > > > > > >> take longer > > > > > > >>>>>>> to arrive at consensus on these semantics, and would > > > > > > >> defer that > > > > > > >>>>>>> discussion for now. > > > > > > >>>>>>> Thanks - Patrick > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>> > > > > -------------------------------------------------------------- > > > > > > >>>>> ---------- > > > > > > >>>>>>> *From:* Chris Hoffman > > > > [mailto:chris.hoffman@berkeley.edu] > > > > > > >>>>>>> *Sent:* Wednesday, August 24, 2011 9:01 AM > > > > > > >>>>>>> *To:* Patrick Schmitz > > > > > > >>>>>>> *Cc:* 'Christopher Pott'; 'Aron Roberts'; > > > > > > >>>>>>> talk@lists.collectionspace.org > > > > > > >>>>>>> *Subject:* Re: [Talk] what's the best way to > > > > > > >>>>>>> represent > > > > > > >>>>> structured > > > > > > >>>>>>> dates? > > > > > > >>>>>>> > > > > > > >>>>>>> Hi Patrick, I think it's fair to say there's no > > > > > > >>>>> consensus because > > > > > > >>>>>>> there's been no discussion :-) It's possible that > > > > > > >>>>>>> each > > > > > > >>>>> customer will > > > > > > >>>>>>> want to use structured dates differently, > > which will > > > > > > >>>>> make it hard to > > > > > > >>>>>>> add any intelligence or even predictability to > > > > > > >>>>>>> things > > > > > > >>>>> like search. > > > > > > >>>>>>> However, I also think that if a > > significant number > > > > > > >>>>>>> of deploying > > > > > > >>>>>>> institutions can come to some consensus on best > > > > > > >> ways to use > > > > > > >>>>>>> and > > > > > > >>>>>>> search structured dates, then many > > institutions will > > > > > > >>>>> choose to adopt > > > > > > >>>>>>> the same best practices. > > > > > > >>>>>>> > > > > > > >>>>>>> Thanks, > > > > > > >>>>>>> Chris > > > > > > >>>>>>> > > > > > > >>>>>>> On Aug 24, 2011, at 8:55 AM, Patrick > > Schmitz wrote: > > > > > > >>>>>>> > > > > > > >>>>>>>> Aron was out yesterday, and Rick is out for a few > > > > > > >>>>> days. They have > > > > > > >>>>>>>> done a lot of the structured dates work to date. > > > > > > >>>>>>>> The last I heard, Aron was hoping to get > > consensus > > > > > > >>>>> from the users > > > > > > >>>>>>>> about how we map all the text fields to > > actual date > > > > > > >>>>> fields that we > > > > > > >>>>>>>> can reason on (for search, ordering, > > etc). To date, > > > > > > >>>>> there seems to > > > > > > >>>>>>>> be no such consensus, which will preclude search, > > > > > > >>>>> ordering, etc. > > > > > > >>>>>>>> on structured dates. I would think that > > this would > > > > > > >>>>> make structured > > > > > > >>>>>>>> dates quite a bit less useful... > > > > > > >>>>>>>> Perhaps (I hope) my understanding is out > > of date, > > > > > > >>>>>>>> and > > > > > > >>>>> we just need > > > > > > >>>>>>>> to implement the logic. Perhaps Aron can jump in > > > > > > >>>>>>>> this > > > > > > >>>>> AM, when he > > > > > > >>>>>>>> gets in. > > > > > > >>>>>>>> Patrick > > > > > > >>>>>>>> > > > > > > >>>>>>>> > > > > > > >>>>>>>> > > > > > > >>>>> > > > > -------------------------------------------------------------- > > > > > > >>>>> ---------- > > > > > > >>>>>>>> *From:* > > talk-bounces@lists.collectionspace.org > > > > > > >>>>>>>> > > <mailto:talk-bounces@lists.collectionspace.org> > > > > > > >>>>>>>> > > [mailto:talk-bounces@lists.collectionspace.org] > > > > > > >>>>> *On Behalf Of > > > > > > >>>>>>>> *Chris Hoffman > > > > > > >>>>>>>> *Sent:* Tuesday, August 23, 2011 11:05 AM > > > > > > >>>>>>>> *To:* Christopher Pott > > > > > > >>>>>>>> *Cc:* talk@lists.collectionspace.org > > > > > > >>>>>>>> <mailto:talk@lists.collectionspace.org> > > > > > > >>>>>>>> *Subject:* Re: [Talk] what's the best way to > > > > > > represent > > > > > > >>>>>>>> structured dates? > > > > > > >>>>>>>> > > > > > > >>>>>>>> Hi Chris, > > > > > > >>>>>>>> > > > > > > >>>>>>>> Thanks for asking these questions. We've been > > > > > > >>>>> wondering the > > > > > > >>>>>>>> same things. Have you changed any existing > > > > > > >> calendar date > > > > > > >>>>>>>> fields to structured dates or added new > > > > > > >> structured date > > > > > > >>>>>>>> fields? I think this is all quite new, and > > > > > > >>>>> together we might > > > > > > >>>>>>>> be able to come up with some best practices > > > > > > >>>>> (related to data > > > > > > >>>>>>>> migration and data entry). > > > > > > >>>>>>>> > > > > > > >>>>>>>> I suspect that search and advanced search of > > > > > > >>>>> structured date > > > > > > >>>>>>>> fields is going to be an issue for those of > > > > > > >> us deploying > > > > > > >>>>>>>> CollectionSpace. Our users will want > > CSpace to > > > > > > >>>>>>>> be > > > > > > >>>>> smart about > > > > > > >>>>>>>> structured dates, interpreting > > qualifiers, and > > > > > > >>>>>>>> so > > > > > > >>>>> on, but I > > > > > > >>>>>>>> bet that we won't see much as much as we > > > > > > >> would like in > > > > > > >>>>>>>> the > > > > > > >>>>>>>> first version of advanced search. I'm always > > > > > > >> happy to be > > > > > > >>>>>>>> proved wrong! :-) > > > > > > >>>>>>>> > > > > > > >>>>>>>> Chris > > > > > > >>>>>>>> > > > > > > >>>>>>>> On Aug 23, 2011, at 8:48 AM, Christopher > > > > Pott wrote: > > > > > > >>>>>>>> > > > > > > >>>>>>>>> Hi, > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> I'm looking at the structured date > > schema and > > > > > > >>>>> the best way to > > > > > > >>>>>>>>> represent dates migrated from our > > old system. > > > > > > >>>>> Any information > > > > > > >>>>>>>>> about the following would be much > > appreciated.. > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> 1. I see some different/overlapping > > > > > > possibilities for > > > > > > >>>>>>>>> recording an object creation date. For > > > > > > >>>>>>>>> example, > > > > > > >>>>> to represent > > > > > > >>>>>>>>> "1718" (display date) one could > > either store: > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> a) a single date, providing year only (1718) > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> b) an earliest date (01.01.1718) > > with a latest > > > > > > >>>>>>>>> date > > > > > > >>>>>>>>> (31.12.1718) > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> c) a single date (01.01.1718) with a > > > > > > >> qualifier of + 364 > > > > > > >>>>>>>>> days > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Is there any difference in the meaning of > > > > > > >>>>>>>>> these > > > > > > >>>>> three (and > > > > > > >>>>>>>>> would they all be treated equally by > > > > > > services such as > > > > > > >>>>>>>>> advanced search)? Our migration data is > > > > > > >>>>> currently in format > > > > > > >>>>>>>>> 'b' but the other formats are sometimes > > > > > > >>>>> 'friendlier' for new > > > > > > >>>>>>>>> input. > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> 2. How are date "certainties" and > > > > > > >> "qualifiers" used by > > > > > > >>>>>>>>> advanced search? > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> I've specified "1718", as a > > creation date with > > > > > > >>>>>>>>> a > > > > > > >>>>> certainty of > > > > > > >>>>>>>>> "circa". Will a search for objects created > > > > > > >>>>> during 1717 result > > > > > > >>>>>>>>> in a hit? > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Or is it necessary to use > > qualifiers (eg. 1718 > > > > > > >>>>> +/- 2 years) > > > > > > >>>>>>>>> to define exact limits for this type of > > > > searching? > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> 3. Is the Date schema locally > > extendable? (We > > > > > > >>>>> need a field to > > > > > > >>>>>>>>> hold an English translation of the display > > > > > > >>>>>>>>> text) > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> 4. Will Advanced search by creation date > > > > > > also search > > > > > > >>>>>>>>> the > > > > > > >>>>>>>>> display date or will it just use the > > > > > > single/earliest > > > > > > >>>>>>>>> and > > > > > > >>>>>>>>> latest date groups? > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Thanks, > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Chris > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > _______________________________________________ > > > > > > >>>>>>>>> Talk mailing list > > > > > > >>>>>>>>> Talk@lists.collectionspace.org > > > > > > >>>>>>>>> <mailto:Talk@lists.collectionspace.org> > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>> > > > > > > >> > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.coll > > > > > > >> ec > > > > > > >>>>>>>>> tionspace.org > > > > > > >>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> _______________________________________________ > > > > > > >>>>>>> Talk mailing list > > > > > > >>>>>>> Talk@lists.collectionspace.org > > > > > > >>>>>>> > > > > > > >>>>> > > > > > > >> > > > > > > > > > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collect > > > > > > >>>>> i > > > > > > >>>>>>> onspace.org > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> _______________________________________________ > > > > > > >>>>>> Talk mailing list > > > > > > >>>>>> Talk@lists.collectionspace.org > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >> > > > > > > > > > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collect > > > > > > >>>>> io > > > > > > >>>>>> nspace.org > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>>> _______________________________________________ > > > > > > >>>>> Talk mailing list > > > > > > >>>>> Talk@lists.collectionspace.org > > > > > > >>>>> > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.c > > > > > > >>>>> ollectionspace.org > > > > > > >>>>> > > > > > > >>>> > > > > > > >>>> > > > > > > >>> > > > > > > >>> _______________________________________________ > > > > > > >>> Talk mailing list > > > > > > >>> Talk@lists.collectionspace.org > > > > > > >>> > > > > > > >> > > > > > > > > > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collect > > > > > > i > > > > > > >> o > > > > > > >>> nspace.org > > > > > > >>> > > > > > > >> > > > > > > >> _______________________________________________ > > > > > > >> Talk mailing list > > > > > > >> Talk@lists.collectionspace.org > > > > > > >> > > http://lists.collectionspace.org/mailman/listinfo/talk_lists. > > > > > > >> c > > > > > > >> ollectionspace.org > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Talk mailing list > > > > > Talk@lists.collectionspace.org > > > > > > > > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.collect > > > > io > > > > > nspa > > > > > ce.org > > > > > > > > _______________________________________________ > > > > Talk mailing list > > > > Talk@lists.collectionspace.org > > > > http://lists.collectionspace.org/mailman/listinfo/talk_lists.c > > > > ollectionspace.org > > > > > > > >
AS
Andrew S. Doran
Tue, Aug 30, 2011 5:54 PM

Couple of observations when entering scientific names into the media
handling table. If I enter Costaria I do not get an alphabetical list of
species so Costaria undulatus is in front of Costaria quadrinervia.

Quite often forms, varieties and subspecies come up before. Most of the
time we will be entering the straight species, so Acer campbellii should
appear at the top of the list and not Acer campbellii var. serratifolia.

Finally for extremely repetitive cataloging the ability to have a sticky
field where the scientific name is carried over into a new record would be
extremely useful, even a keyboard command for individual fields (ALT O –
in some databases I have used).

Andrew

--
Andrew S. Doran
Administrative Curator
University & Jepson Herbaria
University of California, Berkeley
1001 Valley Life Sciences Bldg. MC 2465
Berkeley
CA 94720-2465
USA

Office: 510-643-4344
Herbaria: 510-642-2465
URL: http://ucjeps.berkeley.edu
http://www.facebook.com/ucjeps

Couple of observations when entering scientific names into the media handling table. If I enter Costaria I do not get an alphabetical list of species so Costaria undulatus is in front of Costaria quadrinervia. Quite often forms, varieties and subspecies come up before. Most of the time we will be entering the straight species, so Acer campbellii should appear at the top of the list and not Acer campbellii var. serratifolia. Finally for extremely repetitive cataloging the ability to have a sticky field where the scientific name is carried over into a new record would be extremely useful, even a keyboard command for individual fields (ALT O – in some databases I have used). Andrew -- Andrew S. Doran Administrative Curator University & Jepson Herbaria University of California, Berkeley 1001 Valley Life Sciences Bldg. MC 2465 Berkeley CA 94720-2465 USA Office: 510-643-4344 Herbaria: 510-642-2465 URL: http://ucjeps.berkeley.edu http://www.facebook.com/ucjeps