2019-05-13 Meeting notes

Tim Whitlock (Deactivated)

• No progress has been made. The simplified scripts need updating to remove references to healthshare/rr-hs-test which they originally were downloading from and to. There may be an issue over the NSHBT data being shared as they have raised questions on the 3C project which does a similar form of extract. George Swinnerton will get the scripts sorted this week to automate the process and enable assuming the issues from NHS BT are not really a problem.
• Kings have finally sent through files that allow a full and comparable PV feed to be produced using only RDA feeds.  There have been some issues with the sftp transfer process not picking up the files. This has been resolved and extra sensu checks put in place to check for the issues in future. George Swinnertonstill needs to check the PRE/POST vsalues are being completed correctly but it looks like the current feed could be ready to be used on live and the PV feed switched off. Currently waiting on the expansion of the data content.
• SSG pentest issues. In general SSG are not going to make any changes though they will put some validation in place for patient entered data to restrict potential issues.
• rapolas (Unlicensed) has resolved the outstanding issue from the pentest on international radar.
• George Swinnerton has not tested the use of study groups/ programmemberships on the UKRDC. In theory it should just work and no code change is required but needs testing.
• The NHSBT has been returned after resolving a couple of queries that appear to be due to Guys adding ukt neumbers to their patients but getting some wrong so they incorrectly matched. Fran is following this up and getting corrections made. NHSBT should return the file this week
• Relocation contract negotiations drag on..
• George Swinnerton plans to release v1.1.9 of validation today Monday  /wiki/spaces/RR/pages/1330052950  He will then close the current sprint and start the new one defined last week in the validation meeting.
• rapolas (Unlicensed) has been formaly migrating the RaDaR codebase to python3  and it is currently running on staging and demo in due course after testing this will be put on live.

Tim Whitlock (Deactivated)

The nurture extracts used to provide the graphs on the nurturebiobank.org site have not been working correctly due to memory issues on the server. This is odd as in tests the memory usage is no more than 6Gb. The code base used to create the nurture extract is not the latest where Rapolas has fixed the initial issues with excessive memory usage by not holding all data in memory before writing. This now however has a problem as some results now need to be extracted as a pivot table so meaning with the current implementation of this a lot of data is held in memory. George Swinnertonwill look at merging his code branch and the latest extract code to take advantage of Rapolas' improvements. 

A number of issues with the survey data have been found in the current reports. Most of these concern inconsistent naming Christian Names with and without middle initials. This mainly reflects the form scanned and how the name was entered. It has little impact on the data but shows up because the sql database rather than UKRDC repo is the main source for the data. This is also the cause for multiple copies of the same survey due to the paper copy being scanned more than once. The error correcting interface needs updating to delete the extra copies if the UKRDC reports an duplicate survey error. Currently the interface just clears the error and doesn't update the sql database. The final issue is around incorrect NHS numbers caused by mis-scans and numbers being entered wrong on the survey. This has generally only occurred on patients who have not got any other record in the rregistry database so could not be checked against another source. A new UI function will be needed to correct this involving resubmitting just the patient demographics element for the incorrect patient using the correct NHS number for the NI and the old/incorrect one on the MRN this will then update the record and result (usually) in two ukrdcids for the same NHS number. This should create a workitem but doesn't appear to  so a manual process is currently required to identify the masterrecordids required for the merge call. Ideally this could all be done via a UI.

Resubmitting the Patient demographics appears a good way to quickly correct demographics errors when an corrected original file can not be sent such as the Surveys or if the interface is not submitting updates such as the PV programmemberships.

rapolas (Unlicensed) will look at changing the current YourHealth survey Reporting to use the UKRDC for all data and not just the insignia calculated value. This will then need to be compared against the current MSSQL versions to ensure consistency. This should also resolve the survey name issues as they will all be consistently reported.