I would not think it is possible. Patrick Schmitz would be able to
answer definitively but the segregation of core and domain in the
service layer would suggest that it probably wont work. It is the
repeatability aspect that I think would cause the issues.

Chris

On 01/03/2011 14:55, Christopher Pott wrote:

yeap I can see how that would be tedious and really invalidate the
usefulness of the core schema. However, on the upside it is really easy
to connect those new fields to the existing UI pages so you don't have
to completely change all that. All you need to do is set the <selector>
value in the cspace-config.xml

Now here is an interesting question. is there value in still saving the
values from the title repeatable in both the core schema and in your
domain schema?, effectively duplicating data in the service layer but,
at least in the short term, I am mainly thinking about search and list
view which just returns the primary field in title, which currently
isn't easily configurable to a different field without poking the
service layer schemas (tho hopefully will one day be configured easily).
However, I could see it being potentially easy to connect a core group
and a domain group to the same UI group so that they would both be
filled (as much as their fields allow).

If there is value in this I am happy to set up the logic in the App
layer for it. Or do you think it would be potentially confusing? It does
have the feel of a bodge about it but it also has a feeling that it
might be something that has value.

Chris

On 01/03/2011 20:58, Christopher Pott wrote:

Hi Chris,

On the positive side I suspect the 'Title' group is one of the worst cases :-)

I'd rather not duplicate the data. I think it would create a confusing foundation for another developer to build upon later, especially if they bypass the App layer. For example, other (future) Services layer clients may introduce inconsistencies between the two data sets, or a report designer who directly queries sql could easily select the wrong table and miss some data. I'd much sooner prefer to be able to configure which fields are used as 'key' fields and where they are used. This is necessary for anyone planning to extend the schemas. Is this very difficult to configure?

If your proposal is the only realistic option and you are willing to implement it, then I can help to test it. Would it be necessary to duplicate the entire data set for a repeated group, or could we just duplicate the data (In the case of 'Title' isn't it a single field?) needed to fulfill the required functionality?

/Chris

-----Original Message-----
From: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sent: Wed 3/2/2011 10:41 AM
To: Christopher Pott
Cc: talk@lists.collectionspace.org
Subject: Re: SV: [Talk] adding fields to core schema repeatable groups

yeap I can see how that would be tedious and really invalidate the
usefulness of the core schema. However, on the upside it is really easy
to connect those new fields to the existing UI pages so you don't have
to completely change all that. All you need to do is set the <selector>
value in the cspace-config.xml

Now here is an interesting question. is there value in still saving the
values from the title repeatable in both the core schema and in your
domain schema?, effectively duplicating data in the service layer but,
at least in the short term, I am mainly thinking about search and list
view which just returns the primary field in title, which currently
isn't easily configurable to a different field without poking the
service layer schemas (tho hopefully will one day be configured easily).
However, I could see it being potentially easy to connect a core group
and a domain group to the same UI group so that they would both be
filled (as much as their fields allow).

If there is value in this I am happy to set up the logic in the App
layer for it. Or do you think it would be potentially confusing? It does
have the feel of a bodge about it but it also has a feeling that it
might be something that has value.

Chris

On 01/03/2011 20:58, Christopher Pott wrote:

I can see confusion could ensue.

re: configuration of key fields.
Eveything is in place from my point of view to allow configuration of
key fields, however, the configuration files are not currently being
consumed by the service layer as they are still deciding exactly how
they want configuration to work for them. Tho the movement towards the
unfied configuration file in
services/common/src/main/cspace/config/tenants/collectionspace/service-config.xml
does simplify things. and it is mostly possible to create this file
using a script from the cspace-config.xml and there is the hope that
this could work can be finished and built into the build process so we
remove 'user error' where fields are misspelled, not updated, etcetera
across the 3 levels, but current the configuration in cspace-config.xml
just affects the UI and App and you manually have to update the service
layer config.

So currently what happens is. If you set up different key fields to
those out of the box for search and listing, unless you go in and alter
the service layer by hand, there is an element of fan out in queries as
the app isn't getting everything it needs in the first call to the
service layer, (when the service layer returns the list view of
records), so the app then goes and requests an entire record/procedure
file for each result in order to get the fields that are required by the
UI and defined in cspace-config.xml.

So from your point of view, you do get the results and it does look like
everything is working. It is just slow. And this tends to really affect
the edit page view for an record with multiple related procedures and
records as the App needs to poll each related item to get the data for
the UI rather than just having everything on hand from the list view.
If you are interested you can look at the performance logs that are
written on your system.

Unless you have changed the logging I suspect that the app performance
logs are being written to /tmp/cspace-app-perflog.csv (it should rotate
itself so there is only ever 2 logs kept)
I believe this is structure of the file:
1)Timestamp (local time)
2)Unix epoch time in milliseconds, when the response was sent
3)thread
4)from (all from an app point of view)
5)to (all from an app point of view)
6)requestContext
7)HTTP method (e.g. GET, POST ...) and method.getURI()

How domain vs core fields impacts search however, I don't know. That is
a question for the service boys as the App just sends a very basic query
to them and results come back as if by magic. But once again once the
App receives that result set it might have to poll each individual
result to get all the fields required by the search results view if the
list-items haven't been configured identically to the cspace-config. But
you will get your results.

Is that helpful? I think I got a bit off topic

Chris

On 02/03/2011 12:54, Christopher Pott wrote:

sorry I said service-config.xml I meant tenant-binding.xml I think when
talking about service configing

On 02/03/2011 13:52, Chris Martin wrote:

• Search works across all schemas attached to a document, standard.
• List results fields can come from any schema

So as long as we can describe this in cspace-config, we will transform it to
tenant-bindings, and it will work without fan-out.

We are still pushing this new facility out across all the services (mostly,
lots of typing, not hard work). However, it will be done before too long,
and then there should never be a reason for fan-out on search results, or
other lists of results. That's why we're reworking this area of our code.
Any fan-out will reflect improper configuration, not schema definition or
coding constraints.

Patrick

From: talk-bounces@lists.collectionspace.org
[mailto:talk-bounces@lists.collectionspace.org] On Behalf Of Chris Martin
Sent: Wednesday, March 02, 2011 5:53 AM
To: Christopher Pott
Cc: talk@lists.collectionspace.org
Subject: Re: [Talk] adding fields to core schema repeatable groups

I can see confusion could ensue.

re: configuration of key fields.
Eveything is in place from my point of view to allow configuration of key
fields, however, the configuration files are not currently being consumed by
the service layer as they are still deciding exactly how they want
configuration to work for them. Tho the movement towards the unfied
configuration file in
services/common/src/main/cspace/config/tenants/collectionspace/service-confi
g.xml does simplify things. and it is mostly possible to create this file
using a script from the cspace-config.xml and there is the hope that this
could work can be finished and built into the build process so we remove
'user error' where fields are misspelled, not updated, etcetera across the 3
levels, but current the configuration in cspace-config.xml just affects the
UI and App and you manually have to update the service layer config.

So currently what happens is. If you set up different key fields to those
out of the box for search and listing, unless you go in and alter the
service layer by hand, there is an element of fan out in queries as the app
isn't getting everything it needs in the first call to the service layer,
(when the service layer returns the list view of records), so the app then
goes and requests an entire record/procedure file for each result in order
to get the fields that are required by the UI and defined in
cspace-config.xml.

So from your point of view, you do get the results and it does look like
everything is working. It is just slow. And this tends to really affect the
edit page view for an record with multiple related procedures and records as
the App needs to poll each related item to get the data for the UI rather
than just having everything on hand from the list view.
If you are interested you can look at the performance logs that are written
on your system.

Unless you have changed the logging I suspect that the app performance logs
are being written to /tmp/cspace-app-perflog.csv (it should rotate itself so
there is only ever 2 logs kept)
I believe this is structure of the file:
1)Timestamp (local time)
2)Unix epoch time in milliseconds, when the response was sent
3)thread
4)from (all from an app point of view)
5)to (all from an app point of view)
6)requestContext
7)HTTP method (e.g. GET, POST ...) and method.getURI()

How domain vs core fields impacts search however, I don't know. That is a
question for the service boys as the App just sends a very basic query to
them and results come back as if by magic. But once again once the App
receives that result set it might have to poll each individual result to get
all the fields required by the search results view if the list-items haven't
been configured identically to the cspace-config. But you will get your
results.

Is that helpful? I think I got a bit off topic

Chris

On 02/03/2011 12:54, Christopher Pott wrote:

Hi Chris,

On the positive side I suspect the 'Title' group is one of the worst cases
:-)

I'd rather not duplicate the data. I think it would create a confusing
foundation for another developer to build upon later, especially if they
bypass the App layer. For example, other (future) Services layer clients may
introduce inconsistencies between the two data sets, or a report designer
who directly queries sql could easily select the wrong table and miss some
data. I'd much sooner prefer to be able to configure which fields are used
as 'key' fields and where they are used. This is necessary for anyone
planning to extend the schemas. Is this very difficult to configure?

If your proposal is the only realistic option and you are willing to
implement it, then I can help to test it. Would it be necessary to duplicate
the entire data set for a repeated group, or could we just duplicate the
data (In the case of 'Title' isn't it a single field?) needed to fulfill the
required functionality?

/Chris

-----Original Message-----
From: Chris Martin [mailto:csm22@caret.cam.ac.uk]
Sent: Wed 3/2/2011 10:41 AM
To: Christopher Pott
Cc: talk@lists.collectionspace.org
Subject: Re: SV: [Talk] adding fields to core schema repeatable groups

yeap I can see how that would be tedious and really invalidate the
usefulness of the core schema. However, on the upside it is really easy
to connect those new fields to the existing UI pages so you don't have
to completely change all that. All you need to do is set the <selector>
value in the cspace-config.xml

Now here is an interesting question. is there value in still saving the
values from the title repeatable in both the core schema and in your
domain schema?, effectively duplicating data in the service layer but,
at least in the short term, I am mainly thinking about search and list
view which just returns the primary field in title, which currently
isn't easily configurable to a different field without poking the
service layer schemas (tho hopefully will one day be configured easily).
However, I could see it being potentially easy to connect a core group
and a domain group to the same UI group so that they would both be
filled (as much as their fields allow).

If there is value in this I am happy to set up the logic in the App
layer for it. Or do you think it would be potentially confusing? It does
have the feel of a bodge about it but it also has a feeling that it
might be something that has value.

Chris

On 01/03/2011 20:58, Christopher Pott wrote:

http://lists.collectionspace.org/mailman/listinfo/talk_lists.collectionspace
.org