2020-06-08 Meeting notes

• NHSEngland have decided they require a extract file in the old format whilst they are finalising the new format. George has now sent this. Once Retha has confirmed the format is ok the extract can be switched to the new format.
• James has a query for tracing and will start looking at the processing of the tracing file to create tracing records on the UKRDC next. He will run the tracing file through tracing since it could help with the issues found with the national opt out service rejecting the list of NHS numbers due to some being invalid.
• There was a new dependency warning this morning. Andrew will review this and work out where we are and if there are any issues of concern.
• George hasn't heard back from Fiona about the medications issue. He will follow up and hopefully this can be progressed.
• The pv-extract has now had the rrtstatus added using lookups from the renalreg to translate from numberical codes  to PV rrstatus. This lookup table used by the renal reg database has lookup sused by the UKRDC but currently the code using them has this hard coded rather than using a "code list" This needs to be addressed and the two lists kept in sync. This code just needs testing which should occur this week.
• BCH Have now downloaded the VM and we await feedback which will likely require some changes to the code.
• Opt Out will need to be applied to all out data releases by the end of September. We have a slightly difficult situation in that we hold data un S251 where we rely on the units to ensure the data is anonymised if the patient has opted out. Currently very few patients have done this. Running the renalreg database list through National opt out resulted in 2840 numbers being flagged as opted out. This raises the question of how we handle this as unless the unit manages the anonymisation we will need the NHS number to associate the patient with their previous results. Does out IG/Data collection justifications allow us to keep it unanonymised if we don't release it? National Opt out information can be found  here https://digital.nhs.uk/services/national-data-opt-out
• James continues to work on the covid19 opt out code. Hopefully in the next two weeks the database should finally be working and allow more efficient tracing etc rather than recreating merged lists each week. There were a few issues with satellite codes this week and the files are being retraced as a number of patients have apparently died before they tested positive which could be due to incorrect NHSnumbers being supplied etc. 
• RaDaR most of Fiona's tickets have been resolved and are awaiting review before deployment. THe only one that still needs a proper resolution is the signoff status when a patient is added to the Nurture cohort where it needs to default to 0. This issue is being resolved by running an update query to find and correct these. Andrew will look at the create patient code to see if he can modify the initial value configuration for the Nurture case.  There is a plan to bring RaDaR extracts in line with the rest of the data releases. As part of this it is important that the ability of different roles is understood. The radar page detailing this needs checking and updating as appropriate /wiki/spaces/RAD/pages/1222606951. The Jira tickets need to reflect the release version the corrections are in currently this seems to have been missed.
• The kings feed seems to be back up and running.
• Barts continues to send in NHS numbers with spaces.

Given that Q1,Q2 data for patients covers the period of the covid peak we need to look at ways the data for these quarters could be quickly loaded/used. There are a number of options on the table.

1. Load 2019 data with minimal checks and then load 2020 in the normal way. Acknowledge that the 2019 Quarters need to be further corrected and do this after wards. 
2. Process the raw files to extract the data that is required for analysis rather than load them
3. Load the files in to an empty database to allow early access and then load them properly once the 2019 data has been completed.

Option one is not how the validation is designed to work and could lead to issues and errors. Option 2 is an extension of what was used to create the PHE matching dataset. It would allow access to the data but not be able to provide some fields that depend on previous data. Option 3 is preferred as it allows the main process to follow its normal path whilst giving early access to the 2020 data. This approach would forgo some of the validation on the 2020 data but should be the least error prone.

A number of the libraries used by PV old and haven't been upgraded and thus have potential security issues. SSG have started upgrading these to more recent versions as a result they have hit a requirement that tomcat needs to move >8.5 preferably 9 redhat 6 doesn't support this without subscribing to another package. So it it has been agreed that tomcat will be run from source. In addiion tomcat 9 requires java 1.8 whilst currently the pV servers are running 1.7.

To facilitate this we will install tomcat 9 on the servers in /opt and link it to a /opt/tomcat folder which will be used for the importer we have a separate install in /opt/apache-importer where the latest build is linked to /opt/tomcat-importer as currently happens but from a version in /opt. The upgrade process is to install the new version in the appropriate place (/opt or /opt/apache-importer and link to (/opt/tomcat or /opt/tomcat-importer as appropriate) NB the jar files for PV are under this path so they will need to be copied to the webapps directory and the ownership on the files set to the running user (tomcat or tomcat-importer)