Concept Proposals: a simpler workflow

classic Classic list List threaded Threaded
11 messages Options
Darius Jazayeri-2 Darius Jazayeri-2
Reply | Threaded
Open this post in threaded view
|

Concept Proposals: a simpler workflow

Hi All,

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

I'm particularly interested in the scenario where:
  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms
So, the forms development workflow would basically be:
  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  3. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  4. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  5. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  6. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  7. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.
I think the difference between this and prior work on Concept Proposal is that I'm saying:
1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.
2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

(client-side, for use on forms development machines)
  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.
(server-side)
  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.
It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

-Darius

[hidden email] from OpenMRS Developers' mailing list
Michael Seaton Michael Seaton
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike


On 05/10/2012 02:01 PM, Darius Jazayeri wrote:
Hi All,

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

I'm particularly interested in the scenario where:
  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms
So, the forms development workflow would basically be:
  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  3. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  4. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  5. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  6. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  7. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.
I think the difference between this and prior work on Concept Proposal is that I'm saying:
1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.
2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

(client-side, for use on forms development machines)
  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.
(server-side)
  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.
It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

-Darius

[hidden email] from OpenMRS Developers' mailing list

[hidden email] from OpenMRS Developers' mailing list
Burke Mamlin Burke Mamlin
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

-Burke

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:
Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike



On 05/10/2012 02:01 PM, Darius Jazayeri wrote:
Hi All,

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

I'm particularly interested in the scenario where:
  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms
So, the forms development workflow would basically be:
  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  3. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  4. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  5. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  6. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  7. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.
I think the difference between this and prior work on Concept Proposal is that I'm saying:
1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.
2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

(client-side, for use on forms development machines)
  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.
(server-side)
  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.
It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

-Darius



[hidden email] from OpenMRS Developers' mailing list
Friedman, Roger (CDC/CGH/DGHA) (CTR) Friedman, Roger (CDC/CGH/DGHA) (CTR)
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

In reply to this post by Michael Seaton

Let me note that this is only a particular case of higher-level registries.  AFAIK, at the moment, our approach to higher-level registries is to expose our metadata through REST and let the higher-level registry manipulate it.  I think we should be more supportive, and that the solution for this problem should be taken in the broader context.

 

In the narrower context, I think the better approach would be for the form developer to send the descriptions and id/uuids of his/her proposed concepts to the central authority, which either accepts them or provides substitute concepts with their own id/uuids.  Our task would be to make this substitution easy, given that we are now capable of having concept-valued global properties and attributes, and of packaging concepts into custom data types.

 

In this regard, I developed a protocol for this type of proposal/substitution for use in the HR module last summer.  It worked OK so long as the central authority did not make a mistake; it did not have a means for the central authority to request a substitution except in response to a proposal from the field.

 

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Michael Seaton
Sent: Thursday, May 10, 2012 2:33 PM
To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike


On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Darius Jazayeri-3 Darius Jazayeri-3
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

In reply to this post by Burke Mamlin
I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:
Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

-Burke


On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:
Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike



On 05/10/2012 02:01 PM, Darius Jazayeri wrote:
Hi All,

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

I'm particularly interested in the scenario where:
  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms
So, the forms development workflow would basically be:
  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  3. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  4. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  5. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  6. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  7. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.
I think the difference between this and prior work on Concept Proposal is that I'm saying:
1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.
2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

(client-side, for use on forms development machines)
  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.
(server-side)
  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.
It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

-Darius



[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Yeung, Ada K. Yeung, Ada K.
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

Darius,

Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?

Thanks!

-ada

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM
To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

 

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

 

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

 

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

 

-Burke

 

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike




On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius

 

 


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Darius Jazayeri-3 Darius Jazayeri-3
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

For the use cases I'm familiar with (HFE + MDS) I think we'll be able to write a few lines of code that inspect the XML definition of the form, find any concepts that are referenced by UUID, and replace those with a reference to the MVP/CIEL dictionary. (E.g. we'd need to replace <obs conceptId="aaba432432ba4b2a4"/> with <obs conceptId="MVP:123"/>)

I don't know XForms and Infopath well enough to know how you'd do this, so hopefully Jeremy and Daniel can comment.

(How do you move forms from development to your forms server, or from the forms server to production?)

-Darius

On Thu, May 10, 2012 at 1:32 PM, Yeung, Ada K. <[hidden email]> wrote:

Darius,

Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?

Thanks!

-ada

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM


To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

 

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

 

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

 

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

 

-Burke

 

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike




On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius

 

 


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Lauren Stanisic Lauren Stanisic
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

Darius,

 

This explains how we move our forms/concepts from the forms development server to ampath’s production server.

http://wiki.ampath.or.ke/display/ampath/AMRS+Concept+GOLD

 

Still feeling sort of new to this whole world (so correct me if I get off-base at any point), I have a few gut reactions.  For starters, I’d say that this proposal offers the opportunity for a streamlined and error-resistant process, which is great.

 

For our implementation, there are some practical items I think we’d want to know:

1)      A consideration would be the time it takes for a requested concept to be ready for use.

2)      How any mistakes in created concepts would be handled.

3)      How any edits to sent concept requests would be handled (or if this should altogether be avoided).

4)      Ampath has some concepts in our dictionary that are rather local/specific to Kenya.  For example, we have concepts to handle questions asking a household’s average income per month, and responses are specific to the Kenyan Shilling.  Another example is our dietary questions, which include concepts created to represent traditional Kenyan foods.  In these cases, I suppose we would need to provide any necessary information to the central authority so they understand what they’re creating.

5)      A question:  While the central authority is managing your dictionary for you, will it also be keeping all the concepts they create in their own dictionary?   (say, if the central authority is MVP/CIEL)

 

-lauren

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 4:48 PM
To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

For the use cases I'm familiar with (HFE + MDS) I think we'll be able to write a few lines of code that inspect the XML definition of the form, find any concepts that are referenced by UUID, and replace those with a reference to the MVP/CIEL dictionary. (E.g. we'd need to replace <obs conceptId="aaba432432ba4b2a4"/> with <obs conceptId="MVP:123"/>)

 

I don't know XForms and Infopath well enough to know how you'd do this, so hopefully Jeremy and Daniel can comment.

 

(How do you move forms from development to your forms server, or from the forms server to production?)

 

-Darius

On Thu, May 10, 2012 at 1:32 PM, Yeung, Ada K. <[hidden email]> wrote:

Darius,

Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?

Thanks!

-ada

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM


To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

 

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

 

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

 

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

 

-Burke

 

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike




On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius

 

 


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Lauren Stanisic Lauren Stanisic
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

In reply to this post by Darius Jazayeri-3

Woops.  So I’m realizing that the “authority” would be managed by each implementation, so my questions don’t really apply. Sorry for the misunderstanding! J 

 

From: Lauren Stanisic
Sent: Friday, May 11, 2012 1:38 PM
To: '[hidden email]'; [hidden email]
Subject: RE: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

Darius,

 

This explains how we move our forms/concepts from the forms development server to ampath’s production server.

http://wiki.ampath.or.ke/display/ampath/AMRS+Concept+GOLD

 

Still feeling sort of new to this whole world (so correct me if I get off-base at any point), I have a few gut reactions.  For starters, I’d say that this proposal offers the opportunity for a streamlined and error-resistant process, which is great.

 

For our implementation, there are some practical items I think we’d want to know:

1)      A consideration would be the time it takes for a requested concept to be ready for use.

2)      How any mistakes in created concepts would be handled.

3)      How any edits to sent concept requests would be handled (or if this should altogether be avoided).

4)      Ampath has some concepts in our dictionary that are rather local/specific to Kenya.  For example, we have concepts to handle questions asking a household’s average income per month, and responses are specific to the Kenyan Shilling.  Another example is our dietary questions, which include concepts created to represent traditional Kenyan foods.  In these cases, I suppose we would need to provide any necessary information to the central authority so they understand what they’re creating.

5)      A question:  While the central authority is managing your dictionary for you, will it also be keeping all the concepts they create in their own dictionary?   (say, if the central authority is MVP/CIEL)

 

-lauren

From: [hidden email] [[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 4:48 PM
To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

For the use cases I'm familiar with (HFE + MDS) I think we'll be able to write a few lines of code that inspect the XML definition of the form, find any concepts that are referenced by UUID, and replace those with a reference to the MVP/CIEL dictionary. (E.g. we'd need to replace <obs conceptId="aaba432432ba4b2a4"/> with <obs conceptId="MVP:123"/>)

 

I don't know XForms and Infopath well enough to know how you'd do this, so hopefully Jeremy and Daniel can comment.

 

(How do you move forms from development to your forms server, or from the forms server to production?)

 

-Darius

On Thu, May 10, 2012 at 1:32 PM, Yeung, Ada K. <[hidden email]> wrote:

Darius,

Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?

Thanks!

-ada

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM


To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

 

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

 

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

 

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

 

-Burke

 

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike




On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius

 

 


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list
Andrew Kanter Andrew Kanter
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

Actually, no... in this case the central authority is managing for all implementations... so your questions do hold.

Local concepts could still be managed locally... these should be kept to a minimum and really should be shared. For example, income in kenya shillings would presumably be used by others in Kenya. We just need to make sure that the concepts are defined adequately so there is no confusion. Also, CIEL shares the dictionary for everyone, so once the concept is added, mapped, etc. then anyone can use it...

Andy
 
--------------------
Andrew S. Kanter, MD MPH

Asst. Prof. of Clinical Biomedical Informatics and Clinical Epidemiology
Columbia University
Email: [hidden email]
Mobile: +1 (646) 469-2421
Office: +1 (212) 305-4842
Skype: akanter-ippnw
Yahoo: andy_kanter


From: Lauren Stanisic <[hidden email]>
To: [hidden email]
Sent: Friday, May 11, 2012 2:22 PM
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

Woops.  So I’m realizing that the “authority” would be managed by each implementation, so my questions don’t really apply. Sorry for the misunderstanding! J 
 
From: Lauren Stanisic
Sent: Friday, May 11, 2012 1:38 PM
To: '[hidden email]'; [hidden email]
Subject: RE: [OPENMRS-DEV] Concept Proposals: a simpler workflow
 
Darius,
 
This explains how we move our forms/concepts from the forms development server to ampath’s production server.
 
Still feeling sort of new to this whole world (so correct me if I get off-base at any point), I have a few gut reactions.  For starters, I’d say that this proposal offers the opportunity for a streamlined and error-resistant process, which is great.
 
For our implementation, there are some practical items I think we’d want to know:
1)      A consideration would be the time it takes for a requested concept to be ready for use.
2)      How any mistakes in created concepts would be handled.
3)      How any edits to sent concept requests would be handled (or if this should altogether be avoided).
4)      Ampath has some concepts in our dictionary that are rather local/specific to Kenya.  For example, we have concepts to handle questions asking a household’s average income per month, and responses are specific to the Kenyan Shilling.  Another example is our dietary questions, which include concepts created to represent traditional Kenyan foods.  In these cases, I suppose we would need to provide any necessary information to the central authority so they understand what they’re creating.
5)      A question:  While the central authority is managing your dictionary for you, will it also be keeping all the concepts they create in their own dictionary?   (say, if the central authority is MVP/CIEL)
 
-lauren
From: [hidden email] [[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 4:48 PM
To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow
 
For the use cases I'm familiar with (HFE + MDS) I think we'll be able to write a few lines of code that inspect the XML definition of the form, find any concepts that are referenced by UUID, and replace those with a reference to the MVP/CIEL dictionary. (E.g. we'd need to replace <obs conceptId="aaba432432ba4b2a4"/> with <obs conceptId="MVP:123"/>)
 
I don't know XForms and Infopath well enough to know how you'd do this, so hopefully Jeremy and Daniel can comment.
 
(How do you move forms from development to your forms server, or from the forms server to production?)
 
-Darius
On Thu, May 10, 2012 at 1:32 PM, Yeung, Ada K. <[hidden email]> wrote:
Darius,
Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?
Thanks!
-ada
 
From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM

To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow
 
I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.
 
@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?
 
-Darius
On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:
Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.
 
@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?
 
-Burke
 
On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:
Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike



On 05/10/2012 02:01 PM, Darius Jazayeri wrote:
Hi All,
 
I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.
 
I'm particularly interested in the scenario where:
  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms
So, the forms development workflow would basically be:
  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.
I think the difference between this and prior work on Concept Proposal is that I'm saying:
1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.
2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.
 
All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:
 
(client-side, for use on forms development machines)
  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.
(server-side)
  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.
It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?
 
-Darius
 
 

[hidden email] from OpenMRS Developers' mailing list
 

[hidden email] from OpenMRS Developers' mailing list

[hidden email] from OpenMRS Developers' mailing list
 

[hidden email] from OpenMRS Developers' mailing list

[hidden email] from OpenMRS Developers' mailing list



[hidden email] from OpenMRS Developers' mailing list
Darius Jazayeri-3 Darius Jazayeri-3
Reply | Threaded
Open this post in threaded view
|

Re: Concept Proposals: a simpler workflow

In reply to this post by Lauren Stanisic
Right, in your case the "authority" is yourself, i.e. wherever the AMPATH gold dictionary is.

I think the workflow Ada has described to me in the past is that you would create concept proposals on your gold dictionary server, edit the proposals there until they're finally ready, and then create a concept, marking the proposal as complete. (But this all happens on a single server, IIRC, though the plan was to have a sync mechanism that allows proposals to be created elsewhere.)

I'm suggesting something different: instead of creating proposals on the gold dictionary server, you would create real concepts, but on a different dev server. You could edit them there (and build forms with them on the dev server, even before they've been approved). Then, once they're finally ready, the dev server sends a proposal to gold.

Gold eventually approves the concept, and the dev server overwrites the concept you'd temporarily created with the officially-approved copy of the concept from gold.

I suspect this won't work for AMPATH for two related reasons:
1. The concept_id of the concept you create on the dev server won't be the same as the officially-approved one from gold. So you can't actually do form development on the dev server before the concept is approved on gold.
2. The only mechanism for moving Infopath and XForm forms from dev to gold involves sql dumps.

(I'm not certain about that last point though, so I hope Jeremy and Daniel can comment.)

-Darius

On Fri, May 11, 2012 at 11:22 AM, Lauren Stanisic <[hidden email]> wrote:

Woops.  So I’m realizing that the “authority” would be managed by each implementation, so my questions don’t really apply. Sorry for the misunderstanding! J 

 

From: Lauren Stanisic
Sent: Friday, May 11, 2012 1:38 PM
To: '[hidden email]'; [hidden email]
Subject: RE: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

Darius,

 

This explains how we move our forms/concepts from the forms development server to ampath’s production server.

http://wiki.ampath.or.ke/display/ampath/AMRS+Concept+GOLD

 

Still feeling sort of new to this whole world (so correct me if I get off-base at any point), I have a few gut reactions.  For starters, I’d say that this proposal offers the opportunity for a streamlined and error-resistant process, which is great.

 

For our implementation, there are some practical items I think we’d want to know:

1)      A consideration would be the time it takes for a requested concept to be ready for use.

2)      How any mistakes in created concepts would be handled.

3)      How any edits to sent concept requests would be handled (or if this should altogether be avoided).

4)      Ampath has some concepts in our dictionary that are rather local/specific to Kenya.  For example, we have concepts to handle questions asking a household’s average income per month, and responses are specific to the Kenyan Shilling.  Another example is our dietary questions, which include concepts created to represent traditional Kenyan foods.  In these cases, I suppose we would need to provide any necessary information to the central authority so they understand what they’re creating.

5)      A question:  While the central authority is managing your dictionary for you, will it also be keeping all the concepts they create in their own dictionary?   (say, if the central authority is MVP/CIEL)

 

-lauren

From: [hidden email] [[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 4:48 PM


To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

For the use cases I'm familiar with (HFE + MDS) I think we'll be able to write a few lines of code that inspect the XML definition of the form, find any concepts that are referenced by UUID, and replace those with a reference to the MVP/CIEL dictionary. (E.g. we'd need to replace <obs conceptId="aaba432432ba4b2a4"/> with <obs conceptId="MVP:123"/>)

 

I don't know XForms and Infopath well enough to know how you'd do this, so hopefully Jeremy and Daniel can comment.

 

(How do you move forms from development to your forms server, or from the forms server to production?)

 

-Darius

On Thu, May 10, 2012 at 1:32 PM, Yeung, Ada K. <[hidden email]> wrote:

Darius,

Can you explain what the form developers would need to do for #6 - (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed) under forms development workflow?  For example, if we use infopath and xforms?  Anything needs to be edited on the schema design in addition to edits on the forms?

Thanks!

-ada

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Darius Jazayeri
Sent: Thursday, May 10, 2012 3:57 PM


To: [hidden email]
Subject: Re: [OPENMRS-DEV] Concept Proposals: a simpler workflow

 

I figured that that "reset my dictionary" belongs somewhere other than the concept proposal module. I don't know exactly where though. Currently the MVP/CIEL dictionary is distributed as a big sql dump, and you would "reset" by running that to overwrite your dictionary tables. Recent work on Metadata Sharing is supposed to enable letting you pull in updates from a server like MVP/CIEL, but I don't think this is complete.

 

@Ada/Lauren/Jeremy, I'm really curious to hear whether something along these lines would work for you. I think some parts of the workflow wouldn't be quite the same, since Infopath doesn't work with MDS, but would that minimal functionality for the proposal module solve a use case of yours?

 

-Darius

On Thu, May 10, 2012 at 12:11 PM, Burke Mamlin <[hidden email]> wrote:

Perhaps it's assumed or implied or I overlooked it, but consider adding a method to "reset" your messy dev client to the latest-greatest version of your dictionary from your production machine… so you can rinse & repeat.

 

@Ada/Lauren/Jeremy – would Darius' solution meet AMPATH's needs?

 

-Burke

 

On Thu, May 10, 2012 at 2:32 PM, Michael Seaton <[hidden email]> wrote:

Hi Darius,

I think this is a great solution, and would meet our form development needs very well.  We can certainly talk about potential for developer collaboration on this, though I can't promise anything...

Thanks for thinking this through,
Mike




On 05/10/2012 02:01 PM, Darius Jazayeri wrote:

Hi All,

 

I'm working through a simpler approach to Concept Proposals that what has been attempted several times before, and never finished, and I thought I'd share my thoughts while they're fresh.

 

I'm particularly interested in the scenario where:

  • In the cloud there's a concept authority (in my case MVP/CIEL) who manages your dictionary, and you periodically pull updates from there
  • You have one server that has your official concept dictionary (could be your metadata, forms, or production server)
    • No development work happens directly on this machine. Concept dictionary and forms are developed elsewhere, and imported.
  • You have one or more development machines where you do (potentially-messy) development and testing of forms

So, the forms development workflow would basically be:

  1. On a development machine, starting with your master dictionary, you work on a form. It is expected that you will create a bunch of new concepts, revise them, and delete some of them that were mistakes.
  2. When your form is ready-to-go, you identify all concepts on your form that do not come from the master dictionary (i.e. they were newly-created)
    • with HTML Form Entry this should be easy to automate, by checking whether there are any concept references not in the form of MVP:###. Maybe XForms and Infopath could do something similar.
  1. You send that batch of new concepts up to a web service on the concept authority in the cloud, as proposals. You get back tokens you can use to check the status of your proposals.
  2. (Periodically you ping the concept authority, until all proposals from that batch are resolved.)
  3. You hit the concept authority and download its official versions of the concepts that you created locally, and these replace your locally-created concepts.
    • I hope we can leverage the Metadata Sharing module to do this pretty easily.
  1. (Depending on the form entry technology) You edit your form to refer to the new official versions of the concepts you proposed.
  2. At this point you export the form from your dev machine, and import it into your metadata/forms/production server.

I think the difference between this and prior work on Concept Proposal is that I'm saying:

1. You should do forms development on a separate dev machine whose dictionary is expected to get messy.

2. Instead of creating concept proposals, you create actual concepts, so you can do real testing with them.

 

All this leads me to think that we can produce a minimum viable product of the Concept Proposal module with only these features:

 

(client-side, for use on forms development machines)

  • every time you create a new concept, it is marked as "temporary"
  • you can view a list of all temporary concepts, and delete ones you don't like
  • you can select some temporary concepts and "propose to master dictionary"
  • you can see a list of all your submitted proposals, along with their current status
  • when a proposal has been marked as complete by the server, it will overwrite your local "temporary" concept with the new one officially created, and clear the "temporary" flag.

(server-side)

  • web service for proposing a batch of concepts
  • web service for checking the status of a proposal
  • UI showing a list of all open proposals
  • UI for choosing the action for each item in the batch of proposals
    • Created New Concept (specify the concept)
    • Already Exists (specify the concept) 
    • Rejected (specify the free-text reason)
  • Email notification when a new proposal comes in.

It's possible that some ThoughtWorks developers-in-training might work on this as a project. Or I might propose this as a sprint. What do people think about the approach? In particular, is there anyone out there who finds this approach consistent with their needs, and would contribute some dev time to helping make it happen?

 

-Darius

 

 


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list

 


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list


[hidden email] from OpenMRS Developers' mailing list