Carly,

Thanks Carly.  I understand that this is a work in progress, which is
why I am asking a lot questions to make sure I understand how it is
designed to work.

So if Museum X maintains separate vocabularies for person names
depending on their data source (e.g. ULAN, LCNAF, and a local list),
does that mean the same person (perhaps identified by different
preferred name forms in different vocabularies) may exist in more than
one vocabulary?  If that is the case, in version 2.4 will it be the
case that all those name forms might be included in one person
authority record with one name form designated as preferred and the
rest non-preferred?

Jan

On 4/2/12, Carly Bogen cbogen@movingimage.us wrote:

Here's my view from the tech side:

A given real-world person can be represented with entries (Person, Org, etc.
instances) in multiple vocabularies within an authority. This seems like a
bad practice, but we make no attempt to preclude it, since there are a lot
of "John Smith"s out there, and we (services software) cannot deduplicate or
disambiguate (and such functionality is out of scope for us).

The notion of "preferred" and "non-preferred" is associated with each
possible term/name that is tied to a given Person (Org, etc.) record. This
is important. There is one record in a given vocabulary for a real-world
Person. E.g., Samuel Clemens and Mark Twain are not two Person records in an
"authors" vocabulary - they are two terms listed on one Person record. You
choose the preferred one, and the other is non-preferred.

If Mark Twain appears in ULAN, and also in my local Person vocabulary, it is
possible that the preferred term in ULAN is Mark Twain and in the local one
it is Samuel Clemens. Again, seems like a bad practice, but I describe this
to indicate the difference between what is possible technically and what is
sensible.

Finally, the idea that the various vocabularies are conjoined as one
"Authority" has been available (from the UI perspective) for a long while.
In fact, they can also (through configuration) merge the Organization and
Person authorities into a single virtual authority when doing term
completion.

Nevertheless, when adding a new term, the user MUST select in individual
vocabulary within an individual Authority type (Person, or Org, etc.), to
which the new term is added. It does not make sense to add a new Person to a
virtual "Name Authority".

Furthermore, broader/narrower relations among Persons, Orgs, Taxa, etc. are
constrained to instances in the same vocabulary (a ULAN Person cannot have a
broader concept in a Local Person vocabulary).

Finally, the matter of adding a new term is quite different from specifying
whether it is preferred or non-preferred. If the term does not exist at all,
then a new instance (Person, Org, Taxon, etc.) is created, with the entered
term as the only term, and so the "preferred" term.

*** READ THIS CAREFULLY ***

If you want to add a new non-preferred term, you have to edit the existing
instance (Person, Org, etc.) and add the non-preferred term to that record.

While this last UX seems less than ideal, the only way to make it seamless
would require a workflow like:

1. Type in the new term
2. No matches, so it says "Add to Default Person", "Add to ULAN", (just like
   things are now), PLUS it must also have a new entry for "Add as
   non-preferred term".
3. If they select "Add to Default Person", "Add to ULAN", it works like now.
4. If they select "Add as non-preferred term", it must then present ANOTHER
   term completion widget to select an existing Person, Org, etc. to which the
   new non-preferred term is added.

#4 is clearly more complex. It might be doable, but it will take a lot more
work. The secondary term completion widget would require a special mode to
disallow adding new terms (a read-only mode may already exist). In addition,
however, the app layer must be extended to allow for adding the new
non-preferred term to an existing Person, Org, etc. This is not too hard,
but is additional work that is not currently supported.

Patrick

P.s., the wireframes I saw appear to be way out of date - there is no
additional information shown for each term. The Term info is a repeating
block in 2.4, including language, source, etc.

Jan,

Yes, that's correct.  The same person, even if identified by different
preferred name forms in different vocabularies, may exist in more than one
vocabulary.  However, PT and NPT are designated per authority, not per
vocabulary.  That allows one name form to be designated as preferred and
the rest as non-preferred, regardless of what vocabulary they are in in the
authority.

-Carly

On Mon, Apr 2, 2012 at 2:23 PM, Janice Eklund janice.l.eklund@gmail.comwrote:

--

__

Carly Bogen

Acting Registrar

MUSEUM OF THE MOVING IMAGE

36-01 35 Avenue, Astoria, NY 11106
movingimage.us  718 777 6800
Direct: 718 777 6841

Hi Patrick,

There seems to be a misunderstanding.  I think the key difference from what
you just described is that In the plan for 2.4, both preferred and
not-preferred terms should have their own records.

The first wire frame here
http://wiki.collectionspace.org/display/collectionspace/Wireframes+-+Name+Authorityis
up to date, and indicates how the user can choose whether a term is
preferred or non-preferred, and that they can search the entire authority
(not just the vocabulary) for a term's additional preferred or
non-preferred terms.  It also shows that there can be multiple preferred
terms.  There is no additional information for each term in the wire frame,
because each term has its own record.

I am in the process of making sure everything is ready to go for our Design
& Scope discussion next week - I have posted updated wire frames for most
of the functionality, have created the main JIRAs for each part, and am in
the process of posting the remaining workflows and user stories for us to
go over on Monday.

Hope that clears it up,
Carly

On Mon, Apr 2, 2012 at 3:05 PM, Patrick Schmitz pschmitz@berkeley.eduwrote:

--

__

Carly Bogen

Acting Registrar

MUSEUM OF THE MOVING IMAGE

36-01 35 Avenue, Astoria, NY 11106
movingimage.us  718 777 6800
Direct: 718 777 6841

Uh-oh. We do have a misunderstanding, or perhaps a miscommunication.

What you are describing was our old model, that Megan only agreed to because
UCB thought they needed it. A few months ago we (Jan, ChrisH, and I) went
through it all here, and concluded that it was not a good model, and came up
with a new proposed model. I went through that with Megan, and later with
Angela, and we agreed it was a better approach, and would form the basis for
2.4 work. Jan has been describing the 2.4 Concept Authority in terms of this
revised model (although the 2.3 version is simpler, like the current
authorities).

This new model also forms the basis for Internationalization work that
ChrisP and others so much need.

Perhaps we had better talk, and soon...

From: Carly Bogen [mailto:cbogen@movingimage.us]
Sent: Monday, April 02, 2012 12:49 PM
To: Patrick Schmitz
Cc: Janice Eklund; Angela Spinazze; talk@lists.collectionspace.org
Subject: Re: Predictive text for multiple vocabs

Hi Patrick,

There seems to be a misunderstanding.  I think the key difference from what
you just described is that In the plan for 2.4, both preferred and
not-preferred terms should have their own records.

The first wire frame here
http://wiki.collectionspace.org/display/collectionspace/Wireframes+-+Name+Au
thority is up to date, and indicates how the user can choose whether a term
is preferred or non-preferred, and that they can search the entire authority
(not just the vocabulary) for a term's additional preferred or non-preferred
terms.  It also shows that there can be multiple preferred terms.  There is
no additional information for each term in the wire frame, because each term
has its own record.

I am in the process of making sure everything is ready to go for our Design
& Scope discussion next week - I have posted updated wire frames for most of
the functionality, have created the main JIRAs for each part, and am in the
process of posting the remaining workflows and user stories for us to go
over on Monday.

Hope that clears it up,
Carly

On Mon, Apr 2, 2012 at 3:05 PM, Patrick Schmitz pschmitz@berkeley.edu
wrote:

Here's my view from the tech side:

A given real-world person can be represented with entries (Person, Org, etc.
instances) in multiple vocabularies within an authority. This seems like a
bad practice, but we make no attempt to preclude it, since there are a lot
of "John Smith"s out there, and we (services software) cannot deduplicate or
disambiguate (and such functionality is out of scope for us).

The notion of "preferred" and "non-preferred" is associated with each
possible term/name that is tied to a given Person (Org, etc.) record. This
is important. There is one record in a given vocabulary for a real-world
Person. E.g., Samuel Clemens and Mark Twain are not two Person records in an
"authors" vocabulary - they are two terms listed on one Person record. You
choose the preferred one, and the other is non-preferred.

If Mark Twain appears in ULAN, and also in my local Person vocabulary, it is
possible that the preferred term in ULAN is Mark Twain and in the local one
it is Samuel Clemens. Again, seems like a bad practice, but I describe this
to indicate the difference between what is possible technically and what is
sensible.

Finally, the idea that the various vocabularies are conjoined as one
"Authority" has been available (from the UI perspective) for a long while.
In fact, they can also (through configuration) merge the Organization and
Person authorities into a single virtual authority when doing term
completion.

Nevertheless, when adding a new term, the user MUST select in individual
vocabulary within an individual Authority type (Person, or Org, etc.), to
which the new term is added. It does not make sense to add a new Person to a
virtual "Name Authority".

Furthermore, broader/narrower relations among Persons, Orgs, Taxa, etc. are
constrained to instances in the same vocabulary (a ULAN Person cannot have a
broader concept in a Local Person vocabulary).

Finally, the matter of adding a new term is quite different from specifying
whether it is preferred or non-preferred. If the term does not exist at all,
then a new instance (Person, Org, Taxon, etc.) is created, with the entered
term as the only term, and so the "preferred" term.

*** READ THIS CAREFULLY ***

If you want to add a new non-preferred term, you have to edit the existing
instance (Person, Org, etc.) and add the non-preferred term to that record.

While this last UX seems less than ideal, the only way to make it seamless
would require a workflow like:

1. Type in the new term
2. No matches, so it says "Add to Default Person", "Add to ULAN", (just like
   things are now), PLUS it must also have a new entry for "Add as
   non-preferred term".
3. If they select "Add to Default Person", "Add to ULAN", it works like now.
4. If they select "Add as non-preferred term", it must then present ANOTHER
   term completion widget to select an existing Person, Org, etc. to which the
   new non-preferred term is added.

#4 is clearly more complex. It might be doable, but it will take a lot more
work. The secondary term completion widget would require a special mode to
disallow adding new terms (a read-only mode may already exist). In addition,
however, the app layer must be extended to allow for adding the new
non-preferred term to an existing Person, Org, etc. This is not too hard,
but is additional work that is not currently supported.

Patrick

P.s., the wireframes I saw appear to be way out of date - there is no
additional information shown for each term. The Term info is a repeating
block in 2.4, including language, source, etc.

--

__

Carly Bogen

Acting Registrar

MUSEUM OF THE MOVING IMAGE

36-01 35 Avenue, Astoria, NY 11106

movingimage.us http://movingimage.us/    718 777 6800
Direct: 718 777 6841

Thanks all.  This is very helpful.  I have no problem with being
required to edit an existing instance to add a new non-preferred term.

So if I have a vocabulary called TGN that feeds the Place Authority,
and it is pre-loaded with licensed data from the Getty Vocabulary
Program, in order to add a new term that is NOT in the licensed TGN
data and be able to link it as a narrower context to an existing TGN
licensed term I would have to add it to the TGN vocabulary.  This
approach might prove complicated for anyone licensing a vocabulary
that changes over time and would be subject to  periodic maintenance,
upgrades, etc. from the provider unless there is a way to easily
separate locally added new terms from the licensed terms (Admin status
perhaps?)  Would this also be the case for Associative relationships
(thinking down the development line) like "father of", "spouse of",
"pupil of", etc. for Person names?

Jan

On 4/2/12, Carly Bogen cbogen@movingimage.us wrote:

OK so, there is a miscommunication going on here. And, the good news
is that we have the approach documented thanks to Chris H. and an
email that he sent to the Talk list on February 27th.  I do not recall
seeing it but, it does capture Patrick, what I think you were trying
to describe earlier.  Thank you to Carly for finding it.

Patrick and Jan, does the email below address your concerns?
If so, Carly will update the wireframes to match it.  And, Carly, I'll
take you out to dinner next time I'm in New York to thank you for all
of the work that you have done on this.

Is it also the case then, that it is not possible to allow the use of
non-preferred terms?  And, it is possible to use more than one
Preferred Term (i.e., a preferred term in English, Dutch, French,
etc.). If so, fine, but I just want to clarify that that is the
approach.

Thanks!

Angela

So, implementers do you have any problems with this approach?

Angela

On Apr 2, 2012, at 2:58 PM, Patrick Schmitz wrote:

Thanks Jan -

I should note that it is technically reasonable to allow broader relations
across vocabularies  (but still within the same authority item type). I was
thinking it would not be a good idea to allow it, but I am not even sure it
is enforced now. Seems a little screwy to me, but your use-case is a good
counter-example.

Other relationships can be much freer. E.g., "isEmployedBy" would link from
a Person to an Org.

Patrick

Thanks for finding this, and yes, this is what I discussed with Megan, and
what we were planning to implement in 2.4.

Apologies to all (especially Carly) for shifting the sands beneath her feet
just as Megan was off to care for her other baby ;-)

As to your second question: AIUI from the implementers, there are two modes
that must be supported. A mode is defined for a given implementation,
possibly for a given authority, or vocabulary (not sure what granularity is
needed here - we need to get input from more implementers on this).

Mode 1 allows users to reference non-preferred terms in procedure

records. Here, the termCompletion widget will search for both preferred and
non-preferred terms, and matched non-preferred term instances can be set
into the record value.
2.

Mode 2 does not allow users to reference non-preferred terms in

procedure records. Here, the termCompletion widget will search for both
preferred and non-preferred terms, but matched non-preferred term instances
are mapped to the associated preferred term when set into the record value.
Presumably the termCompletion widget should indicate this to the user,
somehow.

We need to clarify how this will work across the layers, but it should not
require extensive rework in any case. The main thing is for us to define
where the modes are configured, and at what granularity. Here are some
choices, in roughly increasing granularity:

Set it for each procedure (per tenant). Thus, e.g., in Intakes, you

would either use all preferred terms, or you could use any mix of preferred
and non-preferred terms. Each procedure would have its own defined behavior.
2.

Set it for each type of Authority. Thus, e.g., whenever a Person

authority is used in any record, it will either allow non-preferred terms,
or it will not. Each type of authority (Person, Organization,
StorageLocation, Taxonomy, etc.) would have its own specified behavior.
3.

Set it for each vocabulary within an Authority. Thus, e.g., whenever

the ULAN Person authority is used in any record, it will either allow
non-preferred terms, or it will not. Other vocabularies, e.g., the
"LocalPerson" vocabulary  (and of course other types of authority
(Organization, StorageLocation, Taxonomy, etc.)) would have their own
specified behavior.
4.

Set it for each field within a record that uses an authority

(assuming some default). Thus, e.g., in Intakes, the "Current Owner" would
have specified behavior (Mode 1 or Mode 2 above) that could be different
from the behavior of the "Depositor" field.

I have no idea what granularity is needed.

Patrick

From: Angela Spinazze [mailto:ats@atspin.com]
Sent: Monday, April 02, 2012 1:42 PM
To: Patrick Schmitz
Cc: Carly Bogen; Janice Eklund; CollectionSpace Talk List
Subject: Re: Predictive text for multiple vocabs

OK so, there is a miscommunication going on here. And, the good news is that
we have the approach documented thanks to Chris H. and an email that he sent
to the Talk list on February 27th.  I do not recall seeing it but, it does
capture Patrick, what I think you were trying to describe earlier.  Thank
you to Carly for finding it.

Patrick and Jan, does the email below address your concerns?
If so, Carly will update the wireframes to match it.  And, Carly, I'll take
you out to dinner next time I'm in New York to thank you for all of the work
that you have done on this.

Is it also the case then, that it is not possible to allow the use of
non-preferred terms?  And, it is possible to use more than one Preferred
Term (i.e., a preferred term in English, Dutch, French, etc.). If so, fine,
but I just want to clarify that that is the approach.

Thanks!

Angela

Hello CSpace colleagues,

On the version 2.4 roadmap there is identified work related to handling
preferred and non-preferred terms in authorities.
http://wiki.collectionspace.org/display/collectionspace/Release+2.4

In talking with Patrick Schmitz and Jan Eklund, we want to propose some
refactoring to authorities to make it easier to support preferred and
non-preferred terms.  Some of these decisions would shape how we start work
on  the Concept authority that is being developed by UCB as a contribution
to the core.  For the following authorities this will involve modifying some
of the fields and adding others to the repeating term block.  Deployments
migrating data will need to consider the effects of changed field names.

• Organization
• Place
• Concept

For some other authorities however we would need to change the non-repeating
term block to a repeating one.  Deployments migrating data will need to
consider how to change their scripts to move names into a nested schema.

• Person
• Storage Location
• Taxonomy

Each of these would then share a common repeatable term block made up of the
following fields

• displayName
• shortDisplayName
• source
• sourceTermID
• sourcePage
• termLanguage
• preferred
• preferredForLang
• termType
• qualifier
  Note that these fields will also help with localization.

These would then support specifying preferred and non-preferred terms on a
single term record.  The behaviors permitted in the UI are still to be
defined, but the driving use case here was that in some cases we need to
allow non-preferred terms to be able to type in a non-preferred term and
have the term-completion widget show us the preferred term.  Right now, only
the primary name on a term are included in the authority-tied fields.

Implementers: What do you think about this approach?  It will create some
work for those of us who have already written database migration scripts,
and it might create work for people who have done some customizations to
these authorities.  We'll send out more information as the potential impacts
become more clear.

So, implementers do you have any problems with this approach?

Angela

On Apr 2, 2012, at 2:58 PM, Patrick Schmitz wrote:

Uh-oh. We do have a misunderstanding, or perhaps a miscommunication.

What you are describing was our old model, that Megan only agreed to because
UCB thought they needed it. A few months ago we (Jan, ChrisH, and I) went
through it all here, and concluded that it was not a good model, and came up
with a new proposed model. I went through that with Megan, and later with
Angela, and we agreed it was a better approach, and would form the basis for
2.4 work. Jan has been describing the 2.4 Concept Authority in terms of this
revised model (although the 2.3 version is simpler, like the current
authorities).

This new model also forms the basis for Internationalization work that
ChrisP and others so much need.

Perhaps we had better talk, and soon...

From: Carly Bogen [mailto:cbogen@movingimage.us]
Sent: Monday, April 02, 2012 12:49 PM
To: Patrick Schmitz
Cc: Janice Eklund; Angela Spinazze; talk@lists.collectionspace.org
Subject: Re: Predictive text for multiple vocabs

Hi Patrick,

There seems to be a misunderstanding.  I think the key difference from what
you just described is that In the plan for 2.4, both preferred and
not-preferred terms should have their own records.

The first wire frame here
http://wiki.collectionspace.org/display/collectionspace/Wireframes+-+Name+Au
thority is up to date, and indicates how the user can choose whether a term
is preferred or non-preferred, and that they can search the entire authority
(not just the vocabulary) for a term's additional preferred or non-preferred
terms.  It also shows that there can be multiple preferred terms.  There is
no additional information for each term in the wire frame, because each term
has its own record.

I am in the process of making sure everything is ready to go for our Design
& Scope discussion next week - I have posted updated wire frames for most of
the functionality, have created the main JIRAs for each part, and am in the
process of posting the remaining workflows and user stories for us to go
over on Monday.

Hope that clears it up,
Carly

On Mon, Apr 2, 2012 at 3:05 PM, Patrick Schmitz pschmitz@berkeley.edu
wrote:

Here's my view from the tech side:

A given real-world person can be represented with entries (Person, Org, etc.
instances) in multiple vocabularies within an authority. This seems like a
bad practice, but we make no attempt to preclude it, since there are a lot
of "John Smith"s out there, and we (services software) cannot deduplicate or
disambiguate (and such functionality is out of scope for us).

The notion of "preferred" and "non-preferred" is associated with each
possible term/name that is tied to a given Person (Org, etc.) record. This
is important. There is one record in a given vocabulary for a real-world
Person. E.g., Samuel Clemens and Mark Twain are not two Person records in an
"authors" vocabulary - they are two terms listed on one Person record. You
choose the preferred one, and the other is non-preferred.

If Mark Twain appears in ULAN, and also in my local Person vocabulary, it is
possible that the preferred term in ULAN is Mark Twain and in the local one
it is Samuel Clemens. Again, seems like a bad practice, but I describe this
to indicate the difference between what is possible technically and what is
sensible.

Finally, the idea that the various vocabularies are conjoined as one
"Authority" has been available (from the UI perspective) for a long while.
In fact, they can also (through configuration) merge the Organization and
Person authorities into a single virtual authority when doing term
completion.

Nevertheless, when adding a new term, the user MUST select in individual
vocabulary within an individual Authority type (Person, or Org, etc.), to
which the new term is added. It does not make sense to add a new Person to a
virtual "Name Authority".

Furthermore, broader/narrower relations among Persons, Orgs, Taxa, etc. are
constrained to instances in the same vocabulary (a ULAN Person cannot have a
broader concept in a Local Person vocabulary).

Finally, the matter of adding a new term is quite different from specifying
whether it is preferred or non-preferred. If the term does not exist at all,
then a new instance (Person, Org, Taxon, etc.) is created, with the entered
term as the only term, and so the "preferred" term.

*** READ THIS CAREFULLY ***

If you want to add a new non-preferred term, you have to edit the existing
instance (Person, Org, etc.) and add the non-preferred term to that record.

While this last UX seems less than ideal, the only way to make it seamless
would require a workflow like:

1. Type in the new term
2. No matches, so it says "Add to Default Person", "Add to ULAN", (just like
   things are now), PLUS it must also have a new entry for "Add as
   non-preferred term".
3. If they select "Add to Default Person", "Add to ULAN", it works like now.
4. If they select "Add as non-preferred term", it must then present ANOTHER
   term completion widget to select an existing Person, Org, etc. to which the
   new non-preferred term is added.

#4 is clearly more complex. It might be doable, but it will take a lot more
work. The secondary term completion widget would require a special mode to
disallow adding new terms (a read-only mode may already exist). In addition,
however, the app layer must be extended to allow for adding the new
non-preferred term to an existing Person, Org, etc. This is not too hard,
but is additional work that is not currently supported.

Patrick

P.s., the wireframes I saw appear to be way out of date - there is no
additional information shown for each term. The Term info is a repeating
block in 2.4, including language, source, etc.

--

__
Carly Bogen
Acting Registrar

MUSEUM OF THE MOVING IMAGE
36-01 35 Avenue, Astoria, NY 11106
movingimage.us http://movingimage.us/    718 777 6800
Direct: 718 777 6841

Thanks Patrick. HAVRC (and CED) often add Person, Place, and Concept
terms that, for one reason or another, are not included in the
licensed Getty vocabularies.  These locally added terms are then
linked to licensed terms so they will appear in the hierarchy where we
would have expected to find them.  Each term has a term source so it's
possible to identify the licensed terms when it comes time to
install/incorporate updates.  In a few cases, locally added terms have
been proposed to the Getty as candidate terms and been incorporated in
subsequent licensed updates.  Others remain as local terms that only
exist in the broader hierarchy in the context of the HAVRC database.
So adding the term to the (largely) licensed vocabulary should really
only be an issue if the term source data is missing.  But the new
proposed structure should accommodate that, as long as whoever is
doing the data entry understands that they need to go into that added
record and make sure the term source is correct.

Jan

On 4/2/12, Patrick Schmitz pschmitz@berkeley.edu wrote: