talk@lists.collectionspace.org
WE HAVE SUNSET THIS LISTSERV - Join us at collectionspace@lyrasislists.org
View all threadswhat's the best way to represent structured dates?
Hi Chris,
No, we've not yet changed any exisiting calendar dates to structured
dates, although we feel most "non administritive" date fields would be
better served by structured dates (though this distinction is not always
straightforward).
I'm still uncertain about this area - presumeably we can't just replace
a calendar date from the core schema but would have to create a
structured date extension field and use that instead? And then would we
be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same
things. Have you changed any existing calendar date fields to
structured dates or added new structured date fields? I think this is
all quite new, and together we might be able to come up with some best
practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is
going to be an issue for those of us deploying CollectionSpace. Our
users will want CSpace to be smart about structured dates, interpreting
qualifiers, and so on, but I bet that we won't see much as much as we
would like in the first version of advanced search. I'm always happy to
be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I'm looking at the structured date schema and the best way to represent
dates migrated from our old system. Any information about the following
would be much appreciated....
- I see some different/overlapping possibilities for recording an
object creation date. For example, to represent "1718" (display date)
one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they
all be treated equally by services such as advanced search)? Our
migration data is currently in format 'b' but the other formats are
sometimes 'friendlier' for new input.
- How are date "certainties" and "qualifiers" used by advanced search?
I've specified "1718", as a creation date with a certainty of "circa".
Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define
exact limits for this type of searching?
-
Is the Date schema locally extendable? (We need a field to hold an
English translation of the display text) -
Will Advanced search by creation date also search the display date or
will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org
Hi Chris,
We'll be doing the same thing to a lot of dates -- making them structured by changing them or replacing them. We've been told it's not hard to change the type, but we don't have much experience with it yet. As we learn more, I'll ask the team to post some tips and tricks.
Thanks,
Chris
On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:
Hi Chris,
No, we’ve not yet changed any exisiting calendar dates to structured dates, although we feel most “non administritive” date fields would be better served by structured dates (though this distinction is not always straightforward).
I’m still uncertain about this area - presumeably we can’t just replace a calendar date from the core schema but would have to create a structured date extension field and use that instead? And then would we be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same things. Have you changed any existing calendar date fields to structured dates or added new structured date fields? I think this is all quite new, and together we might be able to come up with some best practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is going to be an issue for those of us deploying CollectionSpace. Our users will want CSpace to be smart about structured dates, interpreting qualifiers, and so on, but I bet that we won't see much as much as we would like in the first version of advanced search. I'm always happy to be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I’m looking at the structured date schema and the best way to represent dates migrated from our old system. Any information about the following would be much appreciated….
- I see some different/overlapping possibilities for recording an object creation date. For example, to represent “1718” (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they all be treated equally by services such as advanced search)? Our migration data is currently in format ‘b’ but the other formats are sometimes ‘friendlier’ for new input.
-
How are date “certainties” and “qualifiers” used by advanced search?
I’ve specified “1718”, as a creation date with a certainty of “circa”. Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact limits for this type of searching? -
Is the Date schema locally extendable? (We need a field to hold an English translation of the display text)
-
Will Advanced search by creation date also search the display date or will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
I'll try to get a quick read from the paleo folks. I suspect that #1 will be the way to go. Maybe they just have a number field that has the units of "Years BP". Then they probably map ranges of Years BP to different eras such as Carbiniferous. I doubt they need to specify January 21, -2,135,457. But they certainly have to be able to specify "520 MYA plus/minus 250K years in the Cambrian".
Chrs
On Aug 24, 2011, at 2:46 PM, Patrick Schmitz wrote:
So this is what I more or less expected.
I see several alternatives:
- We do something special for Paleo
- We implement our own Calendar support that can get more range out of a
Long storage unit.
2 would be a lot more work, and I am not thrilled with that approach.
I am pretty sure I do not know enough about how Paleo and friends enter
dates, interpret dates, compare them, etc., etc. If they actually use a
different model for these dates, then perhaps 1 is a reasonable approach.
E.g., we could support a different scalar date field that used a different
unit (like years, rather than days), and then they could do searches on that
rather than date fields. A long range of years into the past (E.g., rooted
at 3000 CE) could represent years nearly back to the beginning of our planet
(about 4.2 billion years ago). While astronomic time has a larger range, we
can cross that bridge with a similar approach (e.g., an extension field that
represents decades).
Patrick
-----Original Message-----
From: Michael T. Black [mailto:mtblack@berkeley.edu]
Sent: Wednesday, August 24, 2011 1:59 PM
To: Patrick Schmitz
Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo Pellegatti
Subject: Re: [Talk] what's the best way to represent structured dates?
Hi Patrick,
Just concluded a summit of the chronometric working
group here at the Hearst (or, as we call it, me and Paolo),
and we're happy with a 300Myr limit (BTW-we and presumably
paleo, geology, etc. would call this BP, not BCE). We have a
handful of 50-150Myr BP objects for which we might
conceivably have dates we'd want to enter. However, we can't
think of any cases in which we've got objects older than
300Myr for which the = 300Myr date would have enough
importance to make entry of these dates a requirement for us
at the moment (e.g., we might have a bead made of an
Ordovician brachiopod, but the collection and production
dates would be the critical ones for us, not the date of the
material from which the item was made). I say "at the
moment" because it would be awfully handy to have these dates
entered once we're at the point where we can offer federated
searching across collections/museums.
I can tell you, though, that you'll probably have a
tough time convincing the paleo folks that they don't need to
enter dates for their Carboniferous and earlier collections:
http://www.ucmp.berkeley.edu/science/inverts/carboniferous.php
http://www.ucmp.berkeley.edu/science/inverts/pennsylvanian.php
http://www.ucmp.berkeley.edu/science/inverts/mississippian.php
http://www.ucmp.berkeley.edu/science/inverts/devonian.php
http://www.ucmp.berkeley.edu/science/inverts/silurian.php
http://www.ucmp.berkeley.edu/science/inverts/ordovician.php
http://www.ucmp.berkeley.edu/science/inverts/cambrian.php
http://www.ucmp.berkeley.edu/science/inverts/precambrian.php
Michael
On Aug 24, 2011, at 12:00 PM, Patrick Schmitz wrote:
I think there is lots of potential for fuzzy date
interpretation, and
we can pursue this as a second step.
Next, we need to find out if the 300M yr BCE range will work for
Michael, for Paleo, Essig, et al. I am worried about Paleo
in particular.
Any other folks on this list with collections that go back
a long way?
How much of a limitation would the 300M yr BCE range be?
Thanks - Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of Aron
Roberts
Sent: Wednesday, August 24, 2011 11:39 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund
janice.l.eklund@gmail.com wrote:
I see no problem with the scalar date scenarios listed below.
Thanks much, Patrick, for such a simple, sensible way of creating
single range start and end values, and Jan for your feedback that
this would work for you.
I've had many long discussions with Susan and Glen about
this topic
and their frustrations with 'black box' or otherwise
unsatisfactory
fuzzy date algorithms. Using our current model / schema for
structured dates, with its separate Year/Month/Day fields
for range
start/single date and end date - for which they both get
considerable
credit - and the clear logic you outline below, date range
search can
be transparent and easy to understand.
Note that entry of Year/Month/Day values versus
uncertainty in the
structure date popup is an either/or, as I understand it. If you
want the transparent, easy-to-understand behavior, you
enter a year,
and maybe also a month, or month and day, for the
start/single date,
and
(optionally) for the end date. If you enter uncertainty-related
values, when this is implemented down the road, perhaps a
pluggable
fuzzy date handler would be invoked, a la Dick's suggestion?
Aron
On 8/24/11, Patrick Schmitz pschmitz@berkeley.edu wrote:
Maybe I am missing something, but is there a problem with
the following:
Single date, expressed as "1910". We model this (in earliest and
latest scalar dates) as esd: "Jan 1 1910" and lsd: "Dec 31 1910"
Date range expressed as earliest:"1910", latest:"1915". We
model this
as
esd: "Jan 1 1910" and lsd: "Dec 31 1915"
Date range expressed as earliest:"Jan 1910", latest:"Aug
1910". We
model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910"
The esd and lsd scalar dates are used for search and
ordering, not display.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf
Of Janice
Eklund
Sent: Wednesday, August 24, 2011 10:17 AM
To: Susan Stone
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
I second Susan's concern about false precision. At least
90% of our
dates are year only single dates or date ranges. If you are
considering fuzzy logic down the road, look at the
existing code
carefully. The fuzzy date machine, at least as
implemented at UCB
for the HAVRC, does not handle ancient dates properly.
Jan
On 8/24/11, Susan Stone sstone@socrates.berkeley.edu wrote:
Patrick,
There was quite a bit of discussion of this in our
Wednesday afternoon
meetings at Berkeley that I'm sure Aron will recall. I
don't think
date fields go back far enough in time, so I'd recommend using
integers or floating point numbers and providing the math
in search.
It is also important to make sure that false precision isn't
introduced. If something is dated 1900, the system
shouldn't enter
that as January 1, 1900 unless the uncertainty values can
be used to
differentiate and the display date can be simply 1900.
Susan
On 08/24/2011 09:16 AM, Patrick Schmitz wrote:
Really? I am a little surprised that in all the design
discussions for structured dates this did not come up.
Anyway, I will say that we do not have the time to build
some system
that allows for a lot of custom behavior, or elaborate
heuristics
here (i.e., trying to divine what is intended from a
set of text
fields). I would strongly suggest that we either
- require entry of some actual date fields upon which
the application
can reason,
or
- we define a very simple set of rules for building such
date values
from some of the existing text fields. By very simple,
I mean that
we only consider fields like year, month, day, and
that the rules
produce two scalar dates (early and late), which
either define a
range, or define a point (if they are equal). AIUI, a
structured
date /cannot/ model the idea of "some time/any
time after (or
before) a date".
One further question is the semantic of which date (if
any) will be
used for ordering.
If and when advanced search supports date ranges (which
also implies
support for /before/ and /after/ logic in search), we can
handle the
interval logic associated with the structured dates, but
only if we
have a pair of scalar date values.
Down the road, I do think it would be interesting to
implement some
fuzzier logic on date search, to better support "circa"
logic and the
uncertainty values. However, I suspect that it will
take longer
to arrive at consensus on these semantics, and would
defer that
discussion for now.
Thanks - Patrick
*From:* Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
*Sent:* Wednesday, August 24, 2011 9:01 AM
*To:* Patrick Schmitz
*Cc:* 'Christopher Pott'; 'Aron Roberts';
talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured
dates?
Hi Patrick, I think it's fair to say there's no
consensus because
there's been no discussion :-) It's possible that each
customer will
want to use structured dates differently, which will
make it hard to
add any intelligence or even predictability to things
like search.
However, I also think that if a significant number of
deploying
institutions can come to some consensus on best
ways to use
and
search structured dates, then many institutions will
choose to adopt
the same best practices.
Thanks,
Chris
On Aug 24, 2011, at 8:55 AM, Patrick Schmitz wrote:
Aron was out yesterday, and Rick is out for a few
days. They have
done a lot of the structured dates work to date.
The last I heard, Aron was hoping to get consensus
from the users
about how we map all the text fields to actual date
fields that we
can reason on (for search, ordering, etc). To date,
there seems to
be no such consensus, which will preclude search,
ordering, etc.
on structured dates. I would think that this would
make structured
dates quite a bit less useful...
Perhaps (I hope) my understanding is out of date, and
we just need
to implement the logic. Perhaps Aron can jump in this
AM, when he
gets in.
Patrick
*From:* talk-bounces@lists.collectionspace.org
<mailto:talk-bounces@lists.collectionspace.org>
[mailto:talk-bounces@lists.collectionspace.org]
*On Behalf Of
*Chris Hoffman
*Sent:* Tuesday, August 23, 2011 11:05 AM
*To:* Christopher Pott
*Cc:* talk@lists.collectionspace.org
<mailto:talk@lists.collectionspace.org>
*Subject:* Re: [Talk] what's the best way to
represent
structured dates?
Hi Chris,
Thanks for asking these questions. We've been
wondering the
same things. Have you changed any existing
calendar date
fields to structured dates or added new
structured date
fields? I think this is all quite new, and
together we might
be able to come up with some best practices
(related to data
migration and data entry).
I suspect that search and advanced search of
structured date
fields is going to be an issue for those of
us deploying
CollectionSpace. Our users will want CSpace to be
smart about
structured dates, interpreting qualifiers, and so
on, but I
bet that we won't see much as much as we
would like in
the
first version of advanced search. I'm always
happy to be
proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I'm looking at the structured date schema and
the best way to
represent dates migrated from our old system.
Any information
about the following would be much appreciated..
1. I see some different/overlapping
possibilities for
recording an object creation date. For example,
to represent
"1718" (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date
(31.12.1718)
c) a single date (01.01.1718) with a
qualifier of + 364
days
Is there any difference in the meaning of these
three (and
would they all be treated equally by
services such as
advanced search)? Our migration data is
currently in format
'b' but the other formats are sometimes
'friendlier' for new
input.
2. How are date "certainties" and
"qualifiers" used by
advanced search?
I've specified "1718", as a creation date with a
certainty of
"circa". Will a search for objects created
during 1717 result
in a hit?
Or is it necessary to use qualifiers (eg. 1718
+/- 2 years)
to define exact limits for this type of searching?
3. Is the Date schema locally extendable? (We
need a field to
hold an English translation of the display text)
4. Will Advanced search by creation date
also search
the
display date or will it just use the
single/earliest
and
latest date groups?
Thanks,
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org
<mailto:Talk@lists.collectionspace.org>
io
nspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
o
nspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Hi Chris and Chris,
I've changed a fair number of the dates in the PAHMA deployment to structured dates. And by "changed" I actually mean *replaced.* It was about as straightforward as far as changes to schemas go. I put the new structured dates in a domain schema (anthropology) and left the original calendar dates unchanged, as they're in the common schema. I'll be happy to share any details not obvious from the following. Pretty much all I had to do to change these calendar dates in collectionobjects_common.xsd:
<xs:element name="inscriptionContentDate" type="xs:string"/>
<xs:element name="inscriptionDescriptionDate" type="xs:string"/>
<xs:element name="contentDate" type="xs:string"/>
Was create these structured dates in collectionobjects_anthropology.xsd:
<xs:element name="inscriptionContentDateGroup" type="inscriptionContentDateGroup"/>
<xs:element name="inscriptionDescriptionDateGroup" type="inscriptionDescriptionDateGroup"/>
<xs:element name="contentDateGroup" type="contentDateGroup"/>
<xs:complexType name="inscriptionContentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="inscriptionDescriptionDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="contentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
Then I commented out the old dates in our-tenant-tenant.xml (with mini-build, otherwise in your tenant's xxxx-tenant.xml file):
<field id="inscriptionContentDate" ui-type="date">
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" ui-type="date">
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" ui-type="date" >
<selector>object-description-content-date</selector>
</field>
And I inserted these instead:
<field id="inscriptionContentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionContentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionContentDateGroup</services-tag>
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionDescriptionDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionDescriptionDateGroup</services-tag>
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="contentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>contentDateGroup</services-tag>
<selector>object-description-content-date</selector>
</field>
NB: I didn't have to change/add anything in core-messages.properties, as the new dates have the same ids and selectors as the old dates.
Then I rebuilt/redeployed the App and Service layers and restart both servers and the new dates worked as expected.
The only problem I've had is that you can't [yet] place structured dates into repeating groups (group lists) and have them function properly.
Michael
On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote:
Hi Chris,
We'll be doing the same thing to a lot of dates -- making them structured by changing them or replacing them. We've been told it's not hard to change the type, but we don't have much experience with it yet. As we learn more, I'll ask the team to post some tips and tricks.
Thanks,
Chris
On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:
Hi Chris,
No, we’ve not yet changed any exisiting calendar dates to structured dates, although we feel most “non administritive” date fields would be better served by structured dates (though this distinction is not always straightforward).
I’m still uncertain about this area - presumeably we can’t just replace a calendar date from the core schema but would have to create a structured date extension field and use that instead? And then would we be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same things. Have you changed any existing calendar date fields to structured dates or added new structured date fields? I think this is all quite new, and together we might be able to come up with some best practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is going to be an issue for those of us deploying CollectionSpace. Our users will want CSpace to be smart about structured dates, interpreting qualifiers, and so on, but I bet that we won't see much as much as we would like in the first version of advanced search. I'm always happy to be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I’m looking at the structured date schema and the best way to represent dates migrated from our old system. Any information about the following would be much appreciated….
- I see some different/overlapping possibilities for recording an object creation date. For example, to represent “1718” (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they all be treated equally by services such as advanced search)? Our migration data is currently in format ‘b’ but the other formats are sometimes ‘friendlier’ for new input.
-
How are date “certainties” and “qualifiers” used by advanced search?
I’ve specified “1718”, as a creation date with a certainty of “circa”. Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact limits for this type of searching? -
Is the Date schema locally extendable? (We need a field to hold an English translation of the display text)
-
Will Advanced search by creation date also search the display date or will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Hi Chris -
I believe your interpretations are correct. Note that the first iteration
will ignore fuzzy qualifiers, but we'd like to:
- Allow some adjustment of search based upon these qualifiers
- Allow local specification of what each qualifier means
I think Date should be extensible, but we will run into the issue we have
elsewhere raised, that if the date is in a repeating section, adding another
column to each row is not currently possible (you would have to duplicate
the whole row in your extension schema). We've been talking about solutions
that would be more reasonable for implementers.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of
Christopher Pott
Sent: Thursday, August 25, 2011 5:14 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent structured dates?
Thanks for the responses. Can you tell me if these
interpretations of your responses are correct....
- if we always provide a full (dd-MM-yyyy GG) earliest and
latest date, these will be the dates used by services for
search and ordering - if we provide less information (such as just a year in
earliest date) services will generate an internal earliest
and latest date to cover that period - Services won't be parsing display dates to generate
earliest and latest dates - Services won't be examining 'qualifiers' or 'certainty'
fields so if we rely on these then searching will fail
Is the date schema extensible? We need a way to store some
information with a date, which probably wouldn't be useful to
services but that can help us modify the UI to our requirements.
Our earliest structured date is currently 1600 BC which I
thought was quite old until I read the other posts.
Just out of interest to other implementers, some screen shots
of our planned date UI implementation can be seen here
http://wiki.collectionspace.org/display/collectionspace/Date+a
nd+Time+Use+Cases. It will use a combination of drop down
lists and day/month/year inputs to enable generation of
display dates according to some precise rules (required for
catalogues and signs). It should generate a translation at
the same time, as well as the earliest and latest dates.
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: talk-bounces@lists.collectionspace.org [mailto:talk-
bounces@lists.collectionspace.org] På vegne af Patrick Schmitz
Sendt: 24. august 2011 23:47
Til: 'Michael T. Black'
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
So this is what I more or less expected.
I see several alternatives:
- We do something special for Paleo
- We implement our own Calendar support that can get more
range out
of a Long storage unit.
2 would be a lot more work, and I am not thrilled with that
approach.
I am pretty sure I do not know enough about how Paleo and friends
enter dates, interpret dates, compare them, etc., etc. If they
actually use a different model for these dates, then
perhaps 1 is a reasonable approach.
E.g., we could support a different scalar date field that used a
different unit (like years, rather than days), and then
they could do
searches on that rather than date fields. A long range of
years into
the past (E.g., rooted at 3000 CE) could represent years
nearly back
to the beginning of our planet (about 4.2 billion years ago). While
astronomic time has a larger range, we can cross that bridge with a
similar approach (e.g., an extension field that represents decades).
Patrick
-----Original Message-----
From: Michael T. Black [mailto:mtblack@berkeley.edu]
Sent: Wednesday, August 24, 2011 1:59 PM
To: Patrick Schmitz
Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo Pellegatti
Subject: Re: [Talk] what's the best way to represent
structured dates?
Hi Patrick,
Just concluded a summit of the chronometric working
group here at
the Hearst (or, as we call it, me and Paolo), and we're
happy with a
300Myr limit (BTW-we and presumably paleo, geology, etc.
would call
this BP, not BCE). We have a handful of 50-150Myr BP objects for
which we might conceivably have dates we'd want to enter.
However,
we can't think of any cases in which we've got objects older than
300Myr for which the = 300Myr date would have enough
importance to
make entry of these dates a requirement for us at the
moment (e.g.,
we might have a bead made of an Ordovician brachiopod, but the
collection and production dates would be the critical
ones for us,
not the date of the material from which the item was
made). I say
"at the moment" because it would be awfully handy to have these
dates entered once we're at the point where we can offer
federated
searching across collections/museums.
I can tell you, though, that you'll probably have a tough time
convincing the paleo folks that they don't need to enter
dates for
their Carboniferous and earlier collections:
http://www.ucmp.berkeley.edu/science/inverts/carboniferous.php
http://www.ucmp.berkeley.edu/science/inverts/pennsylvanian.php
http://www.ucmp.berkeley.edu/science/inverts/mississippian.php
http://www.ucmp.berkeley.edu/science/inverts/devonian.php
http://www.ucmp.berkeley.edu/science/inverts/silurian.php
http://www.ucmp.berkeley.edu/science/inverts/ordovician.php
http://www.ucmp.berkeley.edu/science/inverts/cambrian.php
http://www.ucmp.berkeley.edu/science/inverts/precambrian.php
Michael
On Aug 24, 2011, at 12:00 PM, Patrick Schmitz wrote:
I think there is lots of potential for fuzzy date
interpretation, and
we can pursue this as a second step.
Next, we need to find out if the 300M yr BCE range will
work for
Michael, for Paleo, Essig, et al. I am worried about Paleo
in particular.
Any other folks on this list with collections that go back
a long way?
How much of a limitation would the 300M yr BCE range be?
Thanks - Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On
Behalf Of Aron
Roberts
Sent: Wednesday, August 24, 2011 11:39 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund
janice.l.eklund@gmail.com wrote:
I see no problem with the scalar date scenarios listed below.
Thanks much, Patrick, for such a simple, sensible way of
creating single range start and end values, and Jan for your
feedback that this would work for you.
I've had many long discussions with Susan and Glen about
this topic
and their frustrations with 'black box' or otherwise
unsatisfactory
fuzzy date algorithms. Using our current model / schema for
structured dates, with its separate Year/Month/Day fields
for range
start/single date and end date - for which they both get
considerable
credit - and the clear logic you outline below, date range
search can
be transparent and easy to understand.
Note that entry of Year/Month/Day values versus
uncertainty in the
structure date popup is an either/or, as I understand
it. If you
want the transparent, easy-to-understand behavior, you
enter a year,
and maybe also a month, or month and day, for the
start/single date,
and
(optionally) for the end date. If you enter
uncertainty-related
values, when this is implemented down the road, perhaps a
pluggable
fuzzy date handler would be invoked, a la Dick's suggestion?
Aron
On 8/24/11, Patrick Schmitz pschmitz@berkeley.edu wrote:
Maybe I am missing something, but is there a problem with
the following:
Single date, expressed as "1910". We model this (in earliest
and latest scalar dates) as esd: "Jan 1 1910" and
lsd: "Dec 31 1910"
Date range expressed as earliest:"1910", latest:"1915". We
model this
as
esd: "Jan 1 1910" and lsd: "Dec 31 1915"
Date range expressed as earliest:"Jan 1910", latest:"Aug
1910". We
model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910"
The esd and lsd scalar dates are used for search and
ordering, not display.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf
Of Janice
Eklund
Sent: Wednesday, August 24, 2011 10:17 AM
To: Susan Stone
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
I second Susan's concern about false precision. At least
90% of our
dates are year only single dates or date ranges. If you are
considering fuzzy logic down the road, look at the
existing code
carefully. The fuzzy date machine, at least as
implemented at UCB
for the HAVRC, does not handle ancient dates properly.
Jan
On 8/24/11, Susan Stone
sstone@socrates.berkeley.edu wrote:
Patrick,
There was quite a bit of discussion of this in our
Wednesday afternoon
meetings at Berkeley that I'm sure Aron will recall. I
don't think
date fields go back far enough in time, so I'd recommend
using integers or floating point numbers and providing the
math
in search.
It is also important to make sure that false
precision isn't
introduced. If something is dated 1900, the system
shouldn't enter
that as January 1, 1900 unless the uncertainty values can
be used to
differentiate and the display date can be simply 1900.
Susan
On 08/24/2011 09:16 AM, Patrick Schmitz wrote:
Really? I am a little surprised that in all the design
discussions for structured dates this did not come up.
Anyway, I will say that we do not have the time to build
some system
that allows for a lot of custom behavior, or elaborate
heuristics
here (i.e., trying to divine what is intended from a
set of text
fields). I would strongly suggest that we either
- require entry of some actual date fields upon which
the application
can reason,
or
- we define a very simple set of rules for building such
date values
from some of the existing text fields. By very simple,
I mean that
we only consider fields like year, month, day, and
that the rules
produce two scalar dates (early and late), which
either define a
range, or define a point (if they are equal). AIUI, a
structured
date /cannot/ model the idea of "some time/any
time after (or
before) a date".
One further question is the semantic of which date (if
any) will be
used for ordering.
If and when advanced search supports date ranges (which
also implies
support for /before/ and /after/ logic in search), we can
handle the
interval logic associated with the structured dates, but
only if we
have a pair of scalar date values.
Down the road, I do think it would be interesting to
implement some
fuzzier logic on date search, to better support "circa"
logic and the
uncertainty values. However, I suspect that it will
take longer
to arrive at consensus on these semantics, and would
defer that
discussion for now.
Thanks - Patrick
*From:* Chris Hoffman
[mailto:chris.hoffman@berkeley.edu]
*Sent:* Wednesday, August 24, 2011 9:01 AM
*To:* Patrick Schmitz
*Cc:* 'Christopher Pott'; 'Aron Roberts';
talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured
dates?
Hi Patrick, I think it's fair to say there's no
consensus because
there's been no discussion :-) It's possible that each
customer will
want to use structured dates differently, which will
make it hard to
add any intelligence or even predictability to things
like search.
However, I also think that if a significant number of
deploying
institutions can come to some consensus on best
ways to use
and
search structured dates, then many institutions will
choose to adopt
the same best practices.
Thanks,
Chris
On Aug 24, 2011, at 8:55 AM, Patrick Schmitz wrote:
Aron was out yesterday, and Rick is out for a few
days. They have
done a lot of the structured dates work to date.
The last I heard, Aron was hoping to get consensus
from the users
about how we map all the text fields to actual date
fields that we
can reason on (for search, ordering, etc). To date,
there seems to
be no such consensus, which will preclude search,
ordering, etc.
on structured dates. I would think that this would
make structured
dates quite a bit less useful...
Perhaps (I hope) my understanding is out of date, and
we just need
to implement the logic. Perhaps Aron can jump in this
AM, when he
gets in.
Patrick
*From:* talk-bounces@lists.collectionspace.org
<mailto:talk-bounces@lists.collectionspace.org>
[mailto:talk-bounces@lists.collectionspace.org]
*On Behalf Of
*Chris Hoffman
*Sent:* Tuesday, August 23, 2011 11:05 AM
*To:* Christopher Pott
*Cc:* talk@lists.collectionspace.org
<mailto:talk@lists.collectionspace.org>
*Subject:* Re: [Talk] what's the best way to
represent
structured dates?
Hi Chris,
Thanks for asking these questions. We've been
wondering the
same things. Have you changed any existing
calendar date
fields to structured dates or added new
structured date
fields? I think this is all quite new, and
together we might
be able to come up with some best practices
(related to data
migration and data entry).
I suspect that search and advanced search of
structured date
fields is going to be an issue for those of
us deploying
CollectionSpace. Our users will want CSpace to be
smart about
structured dates, interpreting qualifiers, and so
on, but I
bet that we won't see much as much as we
would like in
the
first version of advanced search. I'm always
happy to be
proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher
Pott wrote:
Hi,
I'm looking at the structured date schema and
the best way to
represent dates migrated from our old system.
Any information
about the following would be much appreciated..
1. I see some different/overlapping
possibilities for
recording an object creation date. For example,
to represent
"1718" (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest
date
(31.12.1718)
c) a single date (01.01.1718) with a
qualifier of + 364
days
Is there any difference in the meaning of these
three (and
would they all be treated equally by
services such as
advanced search)? Our migration data is
currently in format
'b' but the other formats are sometimes
'friendlier' for new
input.
2. How are date "certainties" and
"qualifiers" used by
advanced search?
I've specified "1718", as a creation date with a
certainty of
"circa". Will a search for objects created
during 1717 result
in a hit?
Or is it necessary to use qualifiers (eg. 1718
+/- 2 years)
to define exact limits for this type of
searching?
3. Is the Date schema locally extendable? (We
need a field to
hold an English translation of the display text)
4. Will Advanced search by creation date
also search
the
display date or will it just use the
single/earliest
and
latest date groups?
Thanks,
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org
<mailto:Talk@lists.collectionspace.org>
i
o
nspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
nspa
ce.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Hi Michael,
Thanks for sharing this, it's helpful to understand how you've
approached this. What is the problem with the repeating groups? Is it an
issue with your local configuration or is there an oustanding JIRA?
My only fear with replacing core dates, (and some other core fields) is
that other functionality (eg. Search, summary lists) may rely upon
certain core fields being present. But possibly all this can be simply
configured and this is an unjust concern?
Cheers,
Chris
Fra: Michael T. Black [mailto:mtblack@berkeley.edu]
Sendt: 25. august 2011 20:03
Til: Chris Hoffman
Cc: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris and Chris,
I've changed a fair number of the dates in the PAHMA
deployment to structured dates. And by "changed" I actually mean
replaced. It was about as straightforward as far as changes to
schemas go. I put the new structured dates in a domain schema
(anthropology) and left the original calendar dates unchanged, as
they're in the common schema. I'll be happy to share any details not
obvious from the following. Pretty much all I had to do to change these
calendar dates in collectionobjects_common.xsd:
<xs:element name="inscriptionContentDate"
type="xs:string"/>
<xs:element name="inscriptionDescriptionDate"
type="xs:string"/>
<xs:element name="contentDate"
type="xs:string"/>
Was create these structured dates in
collectionobjects_anthropology.xsd:
<xs:element
name="inscriptionContentDateGroup" type="inscriptionContentDateGroup"/>
<xs:element
name="inscriptionDescriptionDateGroup"
type="inscriptionDescriptionDateGroup"/>
<xs:element name="contentDateGroup"
type="contentDateGroup"/>
<xs:complexType
name="inscriptionContentDateGroup">
xs:sequence
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType
name="inscriptionDescriptionDateGroup">
xs:sequence
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="contentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
Then I commented out the old dates in our-tenant-tenant.xml (with
mini-build, otherwise in your tenant's xxxx-tenant.xml file):
<field id="inscriptionContentDate"
ui-type="date">
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate"
ui-type="date">
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" ui-type="date" >
<selector>object-description-content-date</selector>
</field>
And I inserted these instead:
<field id="inscriptionContentDate"
section="anthropology" ui-type="groupfield/structureddate"
primarykey="inscriptionContentDate.dateDisplayDate"
ui-func="cspace.structuredDate" >
<services-tag>inscriptionContentDateGroup</services-tag>
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate"
section="anthropology" ui-type="groupfield/structureddate"
primarykey="inscriptionDescriptionDate.dateDisplayDate"
ui-func="cspace.structuredDate" >
<services-tag>inscriptionDescriptionDateGroup</services-tag>
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate"
section="anthropology" ui-type="groupfield/structureddate"
primarykey="contentDate.dateDisplayDate" ui-func="cspace.structuredDate"
<services-tag>contentDateGroup</services-tag>
<selector>object-description-content-date</selector>
</field>
NB: I didn't have to change/add anything in core-messages.properties, as
the new dates have the same ids and selectors as the old dates.
Then I rebuilt/redeployed the App and Service layers and restart both
servers and the new dates worked as expected.
The only problem I've had is that you can't [yet] place structured dates
into repeating groups (group lists) and have them function properly.
Michael
On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote:
Hi Chris,
We'll be doing the same thing to a lot of dates -- making them
structured by changing them or replacing them. We've been told it's not
hard to change the type, but we don't have much experience with it yet.
As we learn more, I'll ask the team to post some tips and tricks.
Thanks,
Chris
On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:
Hi Chris,
No, we've not yet changed any exisiting calendar dates to structured
dates, although we feel most "non administritive" date fields would be
better served by structured dates (though this distinction is not always
straightforward).
I'm still uncertain about this area - presumeably we can't just replace
a calendar date from the core schema but would have to create a
structured date extension field and use that instead? And then would we
be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same
things. Have you changed any existing calendar date fields to
structured dates or added new structured date fields? I think this is
all quite new, and together we might be able to come up with some best
practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is
going to be an issue for those of us deploying CollectionSpace. Our
users will want CSpace to be smart about structured dates, interpreting
qualifiers, and so on, but I bet that we won't see much as much as we
would like in the first version of advanced search. I'm always happy to
be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I'm looking at the structured date schema and the best way to represent
dates migrated from our old system. Any information about the following
would be much appreciated....
- I see some different/overlapping possibilities for recording an
object creation date. For example, to represent "1718" (display date)
one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they
all be treated equally by services such as advanced search)? Our
migration data is currently in format 'b' but the other formats are
sometimes 'friendlier' for new input.
- How are date "certainties" and "qualifiers" used by advanced search?
I've specified "1718", as a creation date with a certainty of "circa".
Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define
exact limits for this type of searching?
-
Is the Date schema locally extendable? (We need a field to hold an
English translation of the display text) -
Will Advanced search by creation date also search the display date or
will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collections
pace.org
Hi Patrick,
That's great, this information allows us to progress with the migration work for structured date. From what I understand the problem of extending repeating core schemas (including date) is covered by CSPACE-3894 and the remaining Date work by CSPACE-3943.
I appreciate that the long term aim is to allow fuzzy dates and interpretation of qualifiers (is there a JIRA for this?), but I'm concerned this may be beyond the timeframe of our (SMK's) first expected delivery.
I think the first iteration can support our needs, it just places a little more burden on our UI to generate a second date in some cases where a qualifier would have been simpler, but I don't see this as a problem.
Thanks,
Chris
-----Oprindelig meddelelse-----
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 26. august 2011 02:08
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: RE: [Talk] what's the best way to represent structured dates?
Hi Chris -
I believe your interpretations are correct. Note that the first iteration
will ignore fuzzy qualifiers, but we'd like to:
- Allow some adjustment of search based upon these qualifiers
- Allow local specification of what each qualifier means
I think Date should be extensible, but we will run into the issue we have
elsewhere raised, that if the date is in a repeating section, adding
another
column to each row is not currently possible (you would have to duplicate
the whole row in your extension schema). We've been talking about
solutions
that would be more reasonable for implementers.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of
Christopher Pott
Sent: Thursday, August 25, 2011 5:14 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent structured dates?
Thanks for the responses. Can you tell me if these
interpretations of your responses are correct....
- if we always provide a full (dd-MM-yyyy GG) earliest and
latest date, these will be the dates used by services for
search and ordering - if we provide less information (such as just a year in
earliest date) services will generate an internal earliest
and latest date to cover that period - Services won't be parsing display dates to generate
earliest and latest dates - Services won't be examining 'qualifiers' or 'certainty'
fields so if we rely on these then searching will fail
Is the date schema extensible? We need a way to store some
information with a date, which probably wouldn't be useful to
services but that can help us modify the UI to our requirements.
Our earliest structured date is currently 1600 BC which I
thought was quite old until I read the other posts.
Just out of interest to other implementers, some screen shots
of our planned date UI implementation can be seen here
http://wiki.collectionspace.org/display/collectionspace/Date+a
nd+Time+Use+Cases. It will use a combination of drop down
lists and day/month/year inputs to enable generation of
display dates according to some precise rules (required for
catalogues and signs). It should generate a translation at
the same time, as well as the earliest and latest dates.
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: talk-bounces@lists.collectionspace.org [mailto:talk-
bounces@lists.collectionspace.org] På vegne af Patrick Schmitz
Sendt: 24. august 2011 23:47
Til: 'Michael T. Black'
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
So this is what I more or less expected.
I see several alternatives:
- We do something special for Paleo
- We implement our own Calendar support that can get more
range out
of a Long storage unit.
2 would be a lot more work, and I am not thrilled with that
approach.
I am pretty sure I do not know enough about how Paleo and friends
enter dates, interpret dates, compare them, etc., etc. If they
actually use a different model for these dates, then
perhaps 1 is a reasonable approach.
E.g., we could support a different scalar date field that used a
different unit (like years, rather than days), and then
they could do
searches on that rather than date fields. A long range of
years into
the past (E.g., rooted at 3000 CE) could represent years
nearly back
to the beginning of our planet (about 4.2 billion years ago). While
astronomic time has a larger range, we can cross that bridge with a
similar approach (e.g., an extension field that represents decades).
Patrick
-----Original Message-----
From: Michael T. Black [mailto:mtblack@berkeley.edu]
Sent: Wednesday, August 24, 2011 1:59 PM
To: Patrick Schmitz
Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo Pellegatti
Subject: Re: [Talk] what's the best way to represent
structured dates?
Hi Patrick,
Just concluded a summit of the chronometric working
group here at
the Hearst (or, as we call it, me and Paolo), and we're
happy with a
300Myr limit (BTW-we and presumably paleo, geology, etc.
would call
this BP, not BCE). We have a handful of 50-150Myr BP objects for
which we might conceivably have dates we'd want to enter.
However,
we can't think of any cases in which we've got objects older than
300Myr for which the = 300Myr date would have enough
importance to
make entry of these dates a requirement for us at the
moment (e.g.,
we might have a bead made of an Ordovician brachiopod, but the
collection and production dates would be the critical
ones for us,
not the date of the material from which the item was
made). I say
"at the moment" because it would be awfully handy to have these
dates entered once we're at the point where we can offer
federated
searching across collections/museums.
I can tell you, though, that you'll probably have a tough time
convincing the paleo folks that they don't need to enter
dates for
their Carboniferous and earlier collections:
http://www.ucmp.berkeley.edu/science/inverts/carboniferous.php
http://www.ucmp.berkeley.edu/science/inverts/pennsylvanian.php
http://www.ucmp.berkeley.edu/science/inverts/mississippian.php
http://www.ucmp.berkeley.edu/science/inverts/devonian.php
http://www.ucmp.berkeley.edu/science/inverts/silurian.php
http://www.ucmp.berkeley.edu/science/inverts/ordovician.php
http://www.ucmp.berkeley.edu/science/inverts/cambrian.php
http://www.ucmp.berkeley.edu/science/inverts/precambrian.php
Michael
On Aug 24, 2011, at 12:00 PM, Patrick Schmitz wrote:
I think there is lots of potential for fuzzy date
interpretation, and
we can pursue this as a second step.
Next, we need to find out if the 300M yr BCE range will
work for
Michael, for Paleo, Essig, et al. I am worried about Paleo
in particular.
Any other folks on this list with collections that go back
a long way?
How much of a limitation would the 300M yr BCE range be?
Thanks - Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On
Behalf Of Aron
Roberts
Sent: Wednesday, August 24, 2011 11:39 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund
janice.l.eklund@gmail.com wrote:
I see no problem with the scalar date scenarios listed below.
Thanks much, Patrick, for such a simple, sensible way of
creating single range start and end values, and Jan for your
feedback that this would work for you.
I've had many long discussions with Susan and Glen about
this topic
and their frustrations with 'black box' or otherwise
unsatisfactory
fuzzy date algorithms. Using our current model / schema for
structured dates, with its separate Year/Month/Day fields
for range
start/single date and end date - for which they both get
considerable
credit - and the clear logic you outline below, date range
search can
be transparent and easy to understand.
Note that entry of Year/Month/Day values versus
uncertainty in the
structure date popup is an either/or, as I understand
it. If you
want the transparent, easy-to-understand behavior, you
enter a year,
and maybe also a month, or month and day, for the
start/single date,
and
(optionally) for the end date. If you enter
uncertainty-related
values, when this is implemented down the road, perhaps a
pluggable
fuzzy date handler would be invoked, a la Dick's suggestion?
Aron
On 8/24/11, Patrick Schmitz pschmitz@berkeley.edu wrote:
Maybe I am missing something, but is there a problem with
the following:
Single date, expressed as "1910". We model this (in earliest
and latest scalar dates) as esd: "Jan 1 1910" and
lsd: "Dec 31 1910"
Date range expressed as earliest:"1910", latest:"1915". We
model this
as
esd: "Jan 1 1910" and lsd: "Dec 31 1915"
Date range expressed as earliest:"Jan 1910", latest:"Aug
1910". We
model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910"
The esd and lsd scalar dates are used for search and
ordering, not display.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf
Of Janice
Eklund
Sent: Wednesday, August 24, 2011 10:17 AM
To: Susan Stone
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
I second Susan's concern about false precision. At least
90% of our
dates are year only single dates or date ranges. If you are
considering fuzzy logic down the road, look at the
existing code
carefully. The fuzzy date machine, at least as
implemented at UCB
for the HAVRC, does not handle ancient dates properly.
Jan
On 8/24/11, Susan Stone
sstone@socrates.berkeley.edu wrote:
Patrick,
There was quite a bit of discussion of this in our
Wednesday afternoon
meetings at Berkeley that I'm sure Aron will recall. I
don't think
date fields go back far enough in time, so I'd recommend
using integers or floating point numbers and providing the
math
in search.
It is also important to make sure that false
precision isn't
introduced. If something is dated 1900, the system
shouldn't enter
that as January 1, 1900 unless the uncertainty values can
be used to
differentiate and the display date can be simply 1900.
Susan
On 08/24/2011 09:16 AM, Patrick Schmitz wrote:
Really? I am a little surprised that in all the design
discussions for structured dates this did not come up.
Anyway, I will say that we do not have the time to build
some system
that allows for a lot of custom behavior, or elaborate
heuristics
here (i.e., trying to divine what is intended from a
set of text
fields). I would strongly suggest that we either
- require entry of some actual date fields upon which
the application
can reason,
or
- we define a very simple set of rules for building such
date values
from some of the existing text fields. By very simple,
I mean that
we only consider fields like year, month, day, and
that the rules
produce two scalar dates (early and late), which
either define a
range, or define a point (if they are equal). AIUI, a
structured
date /cannot/ model the idea of "some time/any
time after (or
before) a date".
One further question is the semantic of which date (if
any) will be
used for ordering.
If and when advanced search supports date ranges (which
also implies
support for /before/ and /after/ logic in search), we can
handle the
interval logic associated with the structured dates, but
only if we
have a pair of scalar date values.
Down the road, I do think it would be interesting to
implement some
fuzzier logic on date search, to better support "circa"
logic and the
uncertainty values. However, I suspect that it will
take longer
to arrive at consensus on these semantics, and would
defer that
discussion for now.
Thanks - Patrick
*From:* Chris Hoffman
[mailto:chris.hoffman@berkeley.edu]
*Sent:* Wednesday, August 24, 2011 9:01 AM
*To:* Patrick Schmitz
*Cc:* 'Christopher Pott'; 'Aron Roberts';
talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured
dates?
Hi Patrick, I think it's fair to say there's no
consensus because
there's been no discussion :-) It's possible that each
customer will
want to use structured dates differently, which will
make it hard to
add any intelligence or even predictability to things
like search.
However, I also think that if a significant number of
deploying
institutions can come to some consensus on best
ways to use
and
search structured dates, then many institutions will
choose to adopt
the same best practices.
Thanks,
Chris
On Aug 24, 2011, at 8:55 AM, Patrick Schmitz wrote:
Aron was out yesterday, and Rick is out for a few
days. They have
done a lot of the structured dates work to date.
The last I heard, Aron was hoping to get consensus
from the users
about how we map all the text fields to actual date
fields that we
can reason on (for search, ordering, etc). To date,
there seems to
be no such consensus, which will preclude search,
ordering, etc.
on structured dates. I would think that this would
make structured
dates quite a bit less useful...
Perhaps (I hope) my understanding is out of date, and
we just need
to implement the logic. Perhaps Aron can jump in this
AM, when he
gets in.
Patrick
*From:* talk-bounces@lists.collectionspace.org
<mailto:talk-bounces@lists.collectionspace.org>
[mailto:talk-bounces@lists.collectionspace.org]
*On Behalf Of
*Chris Hoffman
*Sent:* Tuesday, August 23, 2011 11:05 AM
*To:* Christopher Pott
*Cc:* talk@lists.collectionspace.org
<mailto:talk@lists.collectionspace.org>
*Subject:* Re: [Talk] what's the best way to
represent
structured dates?
Hi Chris,
Thanks for asking these questions. We've been
wondering the
same things. Have you changed any existing
calendar date
fields to structured dates or added new
structured date
fields? I think this is all quite new, and
together we might
be able to come up with some best practices
(related to data
migration and data entry).
I suspect that search and advanced search of
structured date
fields is going to be an issue for those of
us deploying
CollectionSpace. Our users will want CSpace to be
smart about
structured dates, interpreting qualifiers, and so
on, but I
bet that we won't see much as much as we
would like in
the
first version of advanced search. I'm always
happy to be
proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher
Pott wrote:
Hi,
I'm looking at the structured date schema and
the best way to
represent dates migrated from our old system.
Any information
about the following would be much appreciated..
1. I see some different/overlapping
possibilities for
recording an object creation date. For example,
to represent
"1718" (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest
date
(31.12.1718)
c) a single date (01.01.1718) with a
qualifier of + 364
days
Is there any difference in the meaning of these
three (and
would they all be treated equally by
services such as
advanced search)? Our migration data is
currently in format
'b' but the other formats are sometimes
'friendlier' for new
input.
2. How are date "certainties" and
"qualifiers" used by
advanced search?
I've specified "1718", as a creation date with a
certainty of
"circa". Will a search for objects created
during 1717 result
in a hit?
Or is it necessary to use qualifiers (eg. 1718
+/- 2 years)
to define exact limits for this type of
searching?
3. Is the Date schema locally extendable? (We
need a field to
hold an English translation of the display text)
4. Will Advanced search by creation date
also search
the
display date or will it just use the
single/earliest
and
latest date groups?
Thanks,
Chris
_______________________________________________
Talk mailing list
Talk@lists.collectionspace.org
<mailto:Talk@lists.collectionspace.org>
i
o
nspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
nspa
ce.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org
Hi Chris,
I made the mistake of assuming a Jira had already been filed, since no one seemed surprised to hear that I was having this problem. I didn't find a Jira for this issue, so I created CSPACE-4347.
Michael
On Aug 26, 2011, at 2:13 AM, Christopher Pott wrote:
Hi Michael,
Thanks for sharing this, it’s helpful to understand how you’ve approached this. What is the problem with the repeating groups? Is it an issue with your local configuration or is there an oustanding JIRA?
My only fear with replacing core dates, (and some other core fields) is that other functionality (eg. Search, summary lists) may rely upon certain core fields being present. But possibly all this can be simply configured and this is an unjust concern?
Cheers,
Chris
Fra: Michael T. Black [mailto:mtblack@berkeley.edu]
Sendt: 25. august 2011 20:03
Til: Chris Hoffman
Cc: Christopher Pott; talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris and Chris,
I've changed a fair number of the dates in the PAHMA deployment to structured dates. And by "changed" I actually mean *replaced.* It was about as straightforward as far as changes to schemas go. I put the new structured dates in a domain schema (anthropology) and left the original calendar dates unchanged, as they're in the common schema. I'll be happy to share any details not obvious from the following. Pretty much all I had to do to change these calendar dates in collectionobjects_common.xsd:
<xs:element name="inscriptionContentDate" type="xs:string"/>
<xs:element name="inscriptionDescriptionDate" type="xs:string"/>
<xs:element name="contentDate" type="xs:string"/>
Was create these structured dates in collectionobjects_anthropology.xsd:
<xs:element name="inscriptionContentDateGroup" type="inscriptionContentDateGroup"/>
<xs:element name="inscriptionDescriptionDateGroup" type="inscriptionDescriptionDateGroup"/>
<xs:element name="contentDateGroup" type="contentDateGroup"/>
<xs:complexType name="inscriptionContentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="inscriptionDescriptionDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="contentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
Then I commented out the old dates in our-tenant-tenant.xml (with mini-build, otherwise in your tenant's xxxx-tenant.xml file):
<field id="inscriptionContentDate" ui-type="date">
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" ui-type="date">
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" ui-type="date" >
<selector>object-description-content-date</selector>
</field>
And I inserted these instead:
<field id="inscriptionContentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionContentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionContentDateGroup</services-tag>
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionDescriptionDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionDescriptionDateGroup</services-tag>
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="contentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>contentDateGroup</services-tag>
<selector>object-description-content-date</selector>
</field>
NB: I didn't have to change/add anything in core-messages.properties, as the new dates have the same ids and selectors as the old dates.
Then I rebuilt/redeployed the App and Service layers and restart both servers and the new dates worked as expected.
The only problem I've had is that you can't [yet] place structured dates into repeating groups (group lists) and have them function properly.
Michael
On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote:
Hi Chris,
We'll be doing the same thing to a lot of dates -- making them structured by changing them or replacing them. We've been told it's not hard to change the type, but we don't have much experience with it yet. As we learn more, I'll ask the team to post some tips and tricks.
Thanks,
Chris
On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:
Hi Chris,
No, we’ve not yet changed any exisiting calendar dates to structured dates, although we feel most “non administritive” date fields would be better served by structured dates (though this distinction is not always straightforward).
I’m still uncertain about this area - presumeably we can’t just replace a calendar date from the core schema but would have to create a structured date extension field and use that instead? And then would we be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same things. Have you changed any existing calendar date fields to structured dates or added new structured date fields? I think this is all quite new, and together we might be able to come up with some best practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is going to be an issue for those of us deploying CollectionSpace. Our users will want CSpace to be smart about structured dates, interpreting qualifiers, and so on, but I bet that we won't see much as much as we would like in the first version of advanced search. I'm always happy to be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I’m looking at the structured date schema and the best way to represent dates migrated from our old system. Any information about the following would be much appreciated….
- I see some different/overlapping possibilities for recording an object creation date. For example, to represent “1718” (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they all be treated equally by services such as advanced search)? Our migration data is currently in format ‘b’ but the other formats are sometimes ‘friendlier’ for new input.
-
How are date “certainties” and “qualifiers” used by advanced search?
I’ve specified “1718”, as a creation date with a certainty of “circa”. Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact limits for this type of searching? -
Is the Date schema locally extendable? (We need a field to hold an English translation of the display text)
-
Will Advanced search by creation date also search the display date or will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Oh that's right! Thanks for showing how this is done, Michael.
Chris
On Aug 25, 2011, at 11:03 AM, Michael T. Black wrote:
Hi Chris and Chris,
I've changed a fair number of the dates in the PAHMA deployment to structured dates. And by "changed" I actually mean *replaced.* It was about as straightforward as far as changes to schemas go. I put the new structured dates in a domain schema (anthropology) and left the original calendar dates unchanged, as they're in the common schema. I'll be happy to share any details not obvious from the following. Pretty much all I had to do to change these calendar dates in collectionobjects_common.xsd:
<xs:element name="inscriptionContentDate" type="xs:string"/>
<xs:element name="inscriptionDescriptionDate" type="xs:string"/>
<xs:element name="contentDate" type="xs:string"/>
Was create these structured dates in collectionobjects_anthropology.xsd:
<xs:element name="inscriptionContentDateGroup" type="inscriptionContentDateGroup"/>
<xs:element name="inscriptionDescriptionDateGroup" type="inscriptionDescriptionDateGroup"/>
<xs:element name="contentDateGroup" type="contentDateGroup"/>
<xs:complexType name="inscriptionContentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="inscriptionDescriptionDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="contentDateGroup">
<xs:sequence>
<xs:element name="dateDisplayDate" type="xs:string"/>
<xs:element name="dateAssociation" type="xs:string"/>
<xs:element name="dateEarliestSingleYear" type="xs:string"/>
<xs:element name="dateEarliestSingleMonth" type="xs:string"/>
<xs:element name="dateEarliestSingleDay" type="xs:string"/>
<xs:element name="dateEarliestSingleEra" type="xs:string"/>
<xs:element name="dateEarliestSingleCertainty" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifier" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierValue" type="xs:string"/>
<xs:element name="dateEarliestSingleQualifierUnit" type="xs:string"/>
<xs:element name="dateLatestYear" type="xs:string"/>
<xs:element name="dateLatestMonth" type="xs:string"/>
<xs:element name="dateLatestDay" type="xs:string"/>
<xs:element name="dateLatestEra" type="xs:string"/>
<xs:element name="dateLatestCertainty" type="xs:string"/>
<xs:element name="dateLatestQualifier" type="xs:string"/>
<xs:element name="dateLatestQualifierValue" type="xs:string"/>
<xs:element name="dateLatestQualifierUnit" type="xs:string"/>
<xs:element name="datePeriod" type="xs:string"/>
<xs:element name="dateNote" type="xs:string"/>
</xs:sequence>
</xs:complexType>
Then I commented out the old dates in our-tenant-tenant.xml (with mini-build, otherwise in your tenant's xxxx-tenant.xml file):
<field id="inscriptionContentDate" ui-type="date">
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" ui-type="date">
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" ui-type="date" >
<selector>object-description-content-date</selector>
</field>
And I inserted these instead:
<field id="inscriptionContentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionContentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionContentDateGroup</services-tag>
<selector>object-description-inscription-content-date</selector>
</field>
<field id="inscriptionDescriptionDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="inscriptionDescriptionDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>inscriptionDescriptionDateGroup</services-tag>
<selector>object-description-inscription-description-date</selector>
</field>
<field id="contentDate" section="anthropology" ui-type="groupfield/structureddate" primarykey="contentDate.dateDisplayDate" ui-func="cspace.structuredDate" >
<services-tag>contentDateGroup</services-tag>
<selector>object-description-content-date</selector>
</field>
NB: I didn't have to change/add anything in core-messages.properties, as the new dates have the same ids and selectors as the old dates.
Then I rebuilt/redeployed the App and Service layers and restart both servers and the new dates worked as expected.
The only problem I've had is that you can't [yet] place structured dates into repeating groups (group lists) and have them function properly.
Michael
On Aug 25, 2011, at 10:28 AM, Chris Hoffman wrote:
Hi Chris,
We'll be doing the same thing to a lot of dates -- making them structured by changing them or replacing them. We've been told it's not hard to change the type, but we don't have much experience with it yet. As we learn more, I'll ask the team to post some tips and tricks.
Thanks,
Chris
On Aug 25, 2011, at 6:04 AM, Christopher Pott wrote:
Hi Chris,
No, we’ve not yet changed any exisiting calendar dates to structured dates, although we feel most “non administritive” date fields would be better served by structured dates (though this distinction is not always straightforward).
I’m still uncertain about this area - presumeably we can’t just replace a calendar date from the core schema but would have to create a structured date extension field and use that instead? And then would we be able to perform advanced search on these new dates?
Cheers,
Chris
Fra: Chris Hoffman [mailto:chris.hoffman@berkeley.edu]
Sendt: 23. august 2011 20:05
Til: Christopher Pott
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent structured dates?
Hi Chris,
Thanks for asking these questions. We've been wondering the same things. Have you changed any existing calendar date fields to structured dates or added new structured date fields? I think this is all quite new, and together we might be able to come up with some best practices (related to data migration and data entry).
I suspect that search and advanced search of structured date fields is going to be an issue for those of us deploying CollectionSpace. Our users will want CSpace to be smart about structured dates, interpreting qualifiers, and so on, but I bet that we won't see much as much as we would like in the first version of advanced search. I'm always happy to be proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher Pott wrote:
Hi,
I’m looking at the structured date schema and the best way to represent dates migrated from our old system. Any information about the following would be much appreciated….
- I see some different/overlapping possibilities for recording an object creation date. For example, to represent “1718” (display date) one could either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718) with a latest date (31.12.1718)
c) a single date (01.01.1718) with a qualifier of + 364 days
Is there any difference in the meaning of these three (and would they all be treated equally by services such as advanced search)? Our migration data is currently in format ‘b’ but the other formats are sometimes ‘friendlier’ for new input.
-
How are date “certainties” and “qualifiers” used by advanced search?
I’ve specified “1718”, as a creation date with a certainty of “circa”. Will a search for objects created during 1717 result in a hit?
Or is it necessary to use qualifiers (eg. 1718 +/- 2 years) to define exact limits for this type of searching? -
Is the Date schema locally extendable? (We need a field to hold an English translation of the display text)
-
Will Advanced search by creation date also search the display date or will it just use the single/earliest and latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace.org
Hi Chris -
Perhaps we can talk about a batch process or validation handler that will
produce the dates you want. If you can get the semantic details worked out,
then we can consider a submission to the core project. I would be happy to
work with you on how to make this work.
Patrick
-----Original Message-----
From: Christopher Pott [mailto:Christopher.Pott@smk.dk]
Sent: Friday, August 26, 2011 5:42 AM
To: Patrick Schmitz; talk@lists.collectionspace.org
Subject: SV: [Talk] what's the best way to represent structured dates?
Hi Patrick,
That's great, this information allows us to progress with the
migration work for structured date. From what I understand
the problem of extending repeating core schemas (including
date) is covered by CSPACE-3894 and the remaining Date work
by CSPACE-3943.
I appreciate that the long term aim is to allow fuzzy dates
and interpretation of qualifiers (is there a JIRA for this?),
but I'm concerned this may be beyond the timeframe of our
(SMK's) first expected delivery.
I think the first iteration can support our needs, it just
places a little more burden on our UI to generate a second
date in some cases where a qualifier would have been simpler,
but I don't see this as a problem.
Thanks,
Chris
-----Oprindelig meddelelse-----
Fra: Patrick Schmitz [mailto:pschmitz@berkeley.edu]
Sendt: 26. august 2011 02:08
Til: Christopher Pott; talk@lists.collectionspace.org
Emne: RE: [Talk] what's the best way to represent structured dates?
Hi Chris -
I believe your interpretations are correct. Note that the first
iteration will ignore fuzzy qualifiers, but we'd like to:
- Allow some adjustment of search based upon these qualifiers
- Allow local specification of what each qualifier means
I think Date should be extensible, but we will run into the
issue we
have elsewhere raised, that if the date is in a repeating section,
adding another column to each row is not currently possible
(you would
have to duplicate the whole row in your extension schema).
We've been
talking about solutions that would be more reasonable for
implementers.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of
Christopher Pott
Sent: Thursday, August 25, 2011 5:14 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
Thanks for the responses. Can you tell me if these
interpretations
of your responses are correct....
- if we always provide a full (dd-MM-yyyy GG) earliest and latest
date, these will be the dates used by services for search and
ordering - if we provide less information (such as just a year in earliest
date) services will generate an internal earliest and
latest date to
cover that period
- Services won't be parsing display dates to generate
earliest and
latest dates
- Services won't be examining 'qualifiers' or 'certainty'
fields so if we rely on these then searching will fail
Is the date schema extensible? We need a way to store some
information with a date, which probably wouldn't be useful to
services but that can help us modify the UI to our requirements.
Our earliest structured date is currently 1600 BC which I thought
was quite old until I read the other posts.
Just out of interest to other implementers, some screen
shots of our
planned date UI implementation can be seen here
http://wiki.collectionspace.org/display/collectionspace/Date+a
nd+Time+Use+Cases. It will use a combination of drop down
lists and day/month/year inputs to enable generation of display
dates according to some precise rules (required for
catalogues and
signs). It should generate a translation at the same
time, as well
as the earliest and latest dates.
Cheers,
Chris
-----Oprindelig meddelelse-----
Fra: talk-bounces@lists.collectionspace.org [mailto:talk-
bounces@lists.collectionspace.org] På vegne af Patrick Schmitz
Sendt: 24. august 2011 23:47
Til: 'Michael T. Black'
Cc: talk@lists.collectionspace.org
Emne: Re: [Talk] what's the best way to represent
structured dates?
So this is what I more or less expected.
I see several alternatives:
- We do something special for Paleo
- We implement our own Calendar support that can get more
range out
of a Long storage unit.
2 would be a lot more work, and I am not thrilled with that
approach.
I am pretty sure I do not know enough about how Paleo
and friends
enter dates, interpret dates, compare them, etc., etc. If they
actually use a different model for these dates, then
perhaps 1 is a reasonable approach.
E.g., we could support a different scalar date field
that used a
different unit (like years, rather than days), and then
they could do
searches on that rather than date fields. A long range of
years into
the past (E.g., rooted at 3000 CE) could represent years
nearly back
to the beginning of our planet (about 4.2 billion years ago).
While astronomic time has a larger range, we can cross
that bridge
with a similar approach (e.g., an extension field that
represents decades).
Patrick
-----Original Message-----
From: Michael T. Black [mailto:mtblack@berkeley.edu]
Sent: Wednesday, August 24, 2011 1:59 PM
To: Patrick Schmitz
Cc: Aron Roberts; talk@lists.collectionspace.org; Paolo
Pellegatti
Subject: Re: [Talk] what's the best way to represent
structured dates?
Hi Patrick,
Just concluded a summit of the chronometric working
group here at
the Hearst (or, as we call it, me and Paolo), and we're
happy with a
300Myr limit (BTW-we and presumably paleo, geology, etc.
would call
this BP, not BCE). We have a handful of 50-150Myr BP objects
for which we might conceivably have dates we'd want to enter.
However,
we can't think of any cases in which we've got objects older
than 300Myr for which the = 300Myr date would have enough
importance to
make entry of these dates a requirement for us at the
moment (e.g.,
we might have a bead made of an Ordovician
brachiopod, but the
collection and production dates would be the critical
ones for us,
not the date of the material from which the item was
made). I say
"at the moment" because it would be awfully handy to
have these
dates entered once we're at the point where we can offer
federated
searching across collections/museums.
I can tell you, though, that you'll probably
have a tough time
convincing the paleo folks that they don't need to enter
dates for
their Carboniferous and earlier collections:
http://www.ucmp.berkeley.edu/science/inverts/carboniferous.php
http://www.ucmp.berkeley.edu/science/inverts/pennsylvanian.php
http://www.ucmp.berkeley.edu/science/inverts/mississippian.php
http://www.ucmp.berkeley.edu/science/inverts/devonian.php
http://www.ucmp.berkeley.edu/science/inverts/silurian.php
http://www.ucmp.berkeley.edu/science/inverts/ordovician.php
http://www.ucmp.berkeley.edu/science/inverts/cambrian.php
http://www.ucmp.berkeley.edu/science/inverts/precambrian.php
Michael
On Aug 24, 2011, at 12:00 PM, Patrick Schmitz wrote:
I think there is lots of potential for fuzzy date
interpretation, and
we can pursue this as a second step.
Next, we need to find out if the 300M yr BCE range will
work for
Michael, for Paleo, Essig, et al. I am worried about Paleo
in particular.
Any other folks on this list with collections that go back
a long way?
How much of a limitation would the 300M yr BCE range be?
Thanks - Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On
Behalf Of Aron
Roberts
Sent: Wednesday, August 24, 2011 11:39 AM
To: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
On Wed, Aug 24, 2011 at 11:16 AM, Janice Eklund
janice.l.eklund@gmail.com wrote:
I see no problem with the scalar date scenarios
listed below.
Thanks much, Patrick, for such a simple, sensible way of
creating single range start and end values, and
Jan for your
feedback that this would work for you.
I've had many long discussions with Susan and Glen about
this topic
and their frustrations with 'black box' or otherwise
unsatisfactory
fuzzy date algorithms. Using our current model /
schema for
structured dates, with its separate Year/Month/Day fields
for range
start/single date and end date - for which they both get
considerable
credit - and the clear logic you outline below, date range
search can
be transparent and easy to understand.
Note that entry of Year/Month/Day values versus
uncertainty in the
structure date popup is an either/or, as I understand
it. If you
want the transparent, easy-to-understand behavior, you
enter a year,
and maybe also a month, or month and day, for the
start/single date,
and
(optionally) for the end date. If you enter
uncertainty-related
values, when this is implemented down the road, perhaps a
pluggable
fuzzy date handler would be invoked, a la Dick's
suggestion?
Aron
On 8/24/11, Patrick Schmitz pschmitz@berkeley.edu wrote:
Maybe I am missing something, but is there a problem with
the following:
Single date, expressed as "1910". We model this (in
earliest and latest scalar dates) as esd: "Jan 1
1910" and
lsd: "Dec 31 1910"
Date range expressed as earliest:"1910",
latest:"1915". We
model this
as
esd: "Jan 1 1910" and lsd: "Dec 31 1915"
Date range expressed as earliest:"Jan 1910", latest:"Aug
1910". We
model this as esd: "Jan 1 1910" and lsd: "Aug 31 1910"
The esd and lsd scalar dates are used for search and
ordering, not display.
Patrick
-----Original Message-----
From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org]
On Behalf
Of Janice
Eklund
Sent: Wednesday, August 24, 2011 10:17 AM
To: Susan Stone
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to represent
structured dates?
I second Susan's concern about false precision.
At least
90% of our
dates are year only single dates or date ranges. If you
are considering fuzzy logic down the road, look at the
existing code
carefully. The fuzzy date machine, at least as
implemented at UCB
for the HAVRC, does not handle ancient dates properly.
Jan
On 8/24/11, Susan Stone
sstone@socrates.berkeley.edu wrote:
Patrick,
There was quite a bit of discussion of this in our
Wednesday afternoon
meetings at Berkeley that I'm sure Aron will recall. I
don't think
date fields go back far enough in time, so I'd
recommend
using integers or floating point numbers and providing
the math
in search.
It is also important to make sure that false
precision isn't
introduced. If something is dated 1900, the system
shouldn't enter
that as January 1, 1900 unless the uncertainty
values can
be used to
differentiate and the display date can be simply 1900.
Susan
On 08/24/2011 09:16 AM, Patrick Schmitz wrote:
Really? I am a little surprised that in all
the design
discussions for structured dates this did not come up.
Anyway, I will say that we do not have the
time to build
some system
that allows for a lot of custom behavior, or elaborate
heuristics
here (i.e., trying to divine what is intended from a
set of text
fields). I would strongly suggest that we either
- require entry of some actual date fields
upon which
the application
can reason,
or
- we define a very simple set of rules for building
such
date values
from some of the existing text fields. By very
simple,
I mean that
we only consider fields like year, month, day, and
that the rules
produce two scalar dates (early and late), which
either define a
range, or define a point (if they are
equal). AIUI,
a
structured
date /cannot/ model the idea of "some time/any
time after (or
before) a date".
One further question is the semantic of which date (if
any) will be
used for ordering.
If and when advanced search supports date
ranges (which
also implies
support for /before/ and /after/ logic in search), we
can
handle the
interval logic associated with the structured
dates, but
only if we
have a pair of scalar date values.
Down the road, I do think it would be interesting to
implement some
fuzzier logic on date search, to better
support "circa"
logic and the
uncertainty values. However, I suspect that it will
take longer
to arrive at consensus on these semantics, and would
defer that
discussion for now.
Thanks - Patrick
*From:* Chris Hoffman
[mailto:chris.hoffman@berkeley.edu]
*Sent:* Wednesday, August 24, 2011 9:01 AM
*To:* Patrick Schmitz
*Cc:* 'Christopher Pott'; 'Aron Roberts';
talk@lists.collectionspace.org
Subject: Re: [Talk] what's the best way to
represent
structured
dates?
Hi Patrick, I think it's fair to say there's no
consensus because
there's been no discussion :-) It's possible that
each
customer will
want to use structured dates differently,
which will
make it hard to
add any intelligence or even predictability to
things
like search.
However, I also think that if a
significant number
of deploying
institutions can come to some consensus on best
ways to use
and
search structured dates, then many
institutions will
choose to adopt
the same best practices.
Thanks,
Chris
On Aug 24, 2011, at 8:55 AM, Patrick
Schmitz wrote:
Aron was out yesterday, and Rick is out for a few
days. They have
done a lot of the structured dates work to date.
The last I heard, Aron was hoping to get
consensus
from the users
about how we map all the text fields to
actual date
fields that we
can reason on (for search, ordering,
etc). To date,
there seems to
be no such consensus, which will preclude search,
ordering, etc.
on structured dates. I would think that
this would
make structured
dates quite a bit less useful...
Perhaps (I hope) my understanding is out
of date,
and
we just need
to implement the logic. Perhaps Aron can jump in
this
AM, when he
gets in.
Patrick
*From:*
*On Behalf Of
*Chris Hoffman
*Sent:* Tuesday, August 23, 2011 11:05 AM
*To:* Christopher Pott
*Cc:* talk@lists.collectionspace.org
<mailto:talk@lists.collectionspace.org>
*Subject:* Re: [Talk] what's the best way to
represent
structured dates?
Hi Chris,
Thanks for asking these questions. We've been
wondering the
same things. Have you changed any existing
calendar date
fields to structured dates or added new
structured date
fields? I think this is all quite new, and
together we might
be able to come up with some best practices
(related to data
migration and data entry).
I suspect that search and advanced search of
structured date
fields is going to be an issue for those of
us deploying
CollectionSpace. Our users will want
CSpace to
be
smart about
structured dates, interpreting
qualifiers, and
so
on, but I
bet that we won't see much as much as we
would like in
the
first version of advanced search. I'm always
happy to be
proved wrong! :-)
Chris
On Aug 23, 2011, at 8:48 AM, Christopher
Pott wrote:
Hi,
I'm looking at the structured date
schema and
the best way to
represent dates migrated from our
old system.
Any information
about the following would be much
appreciated..
1. I see some different/overlapping
possibilities for
recording an object creation date. For
example,
to represent
"1718" (display date) one could
either store:
a) a single date, providing year only (1718)
b) an earliest date (01.01.1718)
with a latest
date
(31.12.1718)
c) a single date (01.01.1718) with a
qualifier of + 364
days
Is there any difference in the meaning of
these
three (and
would they all be treated equally by
services such as
advanced search)? Our migration data is
currently in format
'b' but the other formats are sometimes
'friendlier' for new
input.
2. How are date "certainties" and
"qualifiers" used by
advanced search?
I've specified "1718", as a
creation date with
a
certainty of
"circa". Will a search for objects created
during 1717 result
in a hit?
Or is it necessary to use
qualifiers (eg. 1718
+/- 2 years)
to define exact limits for this type of
searching?
3. Is the Date schema locally
extendable? (We
need a field to
hold an English translation of the display
text)
4. Will Advanced search by creation date
also search
the
display date or will it just use the
single/earliest
and
latest date groups?
Thanks,
Chris
Talk mailing list
Talk@lists.collectionspace.org
<mailto:Talk@lists.collectionspace.org>
io
nspa
ce.org
Talk mailing list
Talk@lists.collectionspace.org
http://lists.collectionspace.org/mailman/listinfo/talk_lists.c
ollectionspace.org