Authors: Clare Tagg (Tagg Oram Partnership, www.taggoram.co.uk), Somia Nasim and Peter Goff (The Qualifications & Curriculum Development Agency, www.qcda.org.uk)

The Data

The key to reaching the ambitious and challenging aims of REMS is - data; without collecting and inputting the data into NVivo, the system is non existent (as illustrated in the diagram below).

Data Sources

It was very apparent to the team from the outset that they needed to make a decision on the types of data that should be incorporated into REMS. We needed to decide whether the system should incorporate primary or secondary data. Both were explored in detail and a risk and SWOT analysis was conducted. The recommendation was only to incorporate secondary evidence into the system, with the view to exploring the potential of having primary data once the system was established. The key reasons for doing this were:

• Manageability - a quick scan of the information held and in production, as well as an estimate of what will be produced until 2013, raised concerns about how the vast amount of primary and secondary data will be managed;
• Coding framework and classification - if both types of data went into the system, it would mean the coding framework and classifications applied would become more complex;
• Data Protection Issues - REMS is a system that is intended to be utilised by internal and external colleagues, and as a result it would be very difficult to meet the Data Protection Act and the Research Code of Practice requirements, with primary data;
• Data being misinterpreted - responses data can very easily be misconstrued especially where conclusions are drawn based on a few responses. This would be very difficult to manage and control.

We were still left with a large amount of evidence, as we wanted to incorporate in to the system strategic objectives, policy documents, monitoring, research, evaluation reports and commentary and media articles, from both internal and external stakeholders.

DATE COLLECTION AND INPUT

To monitor and manage the data collected from stakeholders and input it into the system a thorough data collection strategy was put in place, this included:

• Collecting the backlog of evidence from all colleagues, partners and stakeholders.
• Using established databases to draw out the relevant evidence reports.
• Using Google news reader and AMI Software to monitor current and establishing themes and newly published work from RSS feeds.
• Searching stakeholder websites to collect published evidence reports.
• Reviewing education and skills journals for relevant research work.
• Making regular email requests to colleagues for relevant evidence reports.

At this stage the team were faced with some key challenges, these were:

• Dealing with confidential sources - in order to have the most comprehensive system, at times, in particular in the development and exploratory stages of the project, some work may need to be kept confidential for a certain period of time. Some colleagues were concerned with how this will be managed and what measures were in place. As a result, sources were given confidential tags and the word 'confidential' was also written in the source's title. This allows the team to remove these sources easily if necessary.
• Aim of keeping the sources in their original format - one of the main reasons the team moved from N6 to NVivo7 and then NVivo8, was because the software imports Word documents into NVivo, with virtually all of the formatting in place, with in most cases only minor amendments. One challenge has been highly formatted PDFs do not always convert well by NVivo8 and can cause performance degradation. The team is still working on the best strategy for handling long (50-150 page) PDFs.
• Source reliability - As more and more sources were put into the system, it became apparent that some form of source rating was required to make it easy for colleagues to establish the degree to which the source has been subject to the control of research methods. We applied a plus rating system where the higher rating shows a higher degree of research rigour.

Eighteen months into the project REMS contains about 450 sources (see www.taggoram.co.uk/research/cases/REMS) ranging from short media reports of 1 or 2 pages, long research reports of 100 or more pages to policy documents of 150 pages. Many of these sources contain a variety of tables and graphs.

Back to Project Home Page