With the clarity of the New Year, I realized I broke a promise to you DCXL readers… in my post on data policies, I stated that my next post would be about the current state of data management plan evaluation on NSF panels. Although it is a bit late, here’s that post.
My information is from a couple of different sources: a program officer or two at NSF, a few scientists who have served on panels for several different directorates, and some miscellaneous experts in data management plans. In general, they all said about the same thing: we are in early days for data management plans as an NSF requirement, and the process is still evolving. With that in mind, here are a few more specific pieces of information I gathered (note, these should be taken with a grain of salt since this is not the official position of NSF):
- The NSF program officer that leads the panel set the tone for DMP evaluation. Scientists that serve on the proposal review panels generally are not experts in data management or archiving, and therefore are unsure what to look for in DMPs.
- The contents of a data management plan will not tank a proposal unless it is completely absent. Since no one is quite sure what should be in these DMPs, it’s tough to eliminate a good proposal on the basis of its DMP. Overall, DMPs are not currently a part of the merit review process. One person said it very succinctly:
PIs received a slap on the wrist if they had a good proposal with a bad DMP. If it was a bad proposal, the bad DMP was just another nail in the coffin.
- The panelists are merely trying to determine whether at DMP is “adequate”. What does this mean? It generally boils down to two criteria: (1) Is the DMP present? and (2) Does the PI discuss how they will archive the data? Even (2) is up for debate since proposals have made it to the top despite no clear plans for archival, e.g. no mention of where they will archive the data.
- Finally, there is buzz about some knowledgeable PIs using DMPs as a strategic tool. Rather than considering this two-page requirement a burden, they use the DMP as part of their proposal’s narrative. Food for thought.
