Maxis uses data to show the power of RE:Collection for dementia patients

Maxis recently created a data-driven personalised reminiscence mobile app for dementia patients named RE:Collection, enabling users to capture their favourite moments. Family members can also use data and analytics to understand what memory triggers matter to those with dementia. More specifically, the RE:Collection app allows consumers to inspire their loved ones to reminisce over past memories via photo or video triggers; relive stories through AR; and improve the connection between caregiver and seniors by discovering what are the best triggers through insights and analytics.

Launched in line with Chinese New Year, the app was done in collaboration with Kuala Lumpur-based dementia enrichment centre Caring With You, the Alzheimer’s Disease Foundation Malaysia, heads of the Geriatrics Units at hospitals in Kuala Lumpur and Klang, as well as Ministry XR. Ministry XR specialises in the development of spatial computing interfaces such as AR, VR and MR and has worked on tech-driven campaigns with Maxis, such as Deria Takbir and the HuatAR app in 2018.

Concurrently, the telco also rolled out a smart table known as UnforgetTABLE, powered by Maxis 5G, to demonstrate how common household furniture, such as a table, is capable of object recognition and brainwave analytics. When an object with a sensor is placed on top of the smart table, it will replay the memories associated with them on a screen. Also, when users wear a headset sensor, the smart table will display brainwave outputs of their mental state, such as calmness, excitement, boredom, and sadness, on screen. This analytics will enable caregivers to have better understanding of dementia patients.

Using tech to relive happy memories

Malaysia's Alzheimer’s Disease Foundation anticipates that there will be more than 260,000 dementia patients in Malaysia by 2030. Known for humanising technology in previous campaigns, Tai Kam Leong, head of brand and marketing, Maxis told A+M in an interview that it saw an opportunity for technology to aid the particular segment of society that is struggling to hold on to the happy memories this Chinese New Year.

"First, we spoke to experts in the field to understand the landscape and those afflicted by this. We discovered reminiscence therapy, a useful practice between caregivers and patients. Photographs of objects or sceneries are used and experiences triggered by the stimulus, discussed. Reminiscence therapy in its current form is often not personalised, not easy to document, rather inflexible and manual in delivery. Additionally, sometimes these seniors are unable to communicate how they feel in words, accurately," he explained.

Tai added that the solution was sparked by the question of whether Maxis could apply analytics, audio-visual and user-friendly applications to the challenges of reminiscence therapy. Through the RE:Collection app, a simple report is generated when the patient views photograph or video stimulus. The report will inform categories or topics that are important, and the caregivers will have more insight to creating more of the same activities or conversation topics that will stimulate their minds more. Inspired by the potential of 5G, the telco also explored further possibilities such as turning the table from a static object into an interactive, smart piece of technology where moments are revisited.

According to him, careful consideration of a serious topic such as dementia and developing features that may be genuinely useful are some of the challenges faced while working on the RE:Collection app and the UnforgetTable.

The complexity of production timeline for an app and a 5G experiential showcase was time consuming and certainly a challenge to manage.

That said, to help promote both initiatives, Maxis created a fictional story featuring the relationship between a daughter and a father who is suffering dementia. The story follows the journey and pains that both a dementia patient and their loved ones goes through.

Meanwhile, Andrew Yee, CTO of Ministry XR, told A+M that its teams from Singapore and Malaysia comprising data scientists, instrumentation engineers, AI engineers, software developers, UI/UX engineers and designers, were involved in the project. Before the project commenced, the team devoted time on researching the use of electroencephalogram (EEG) sensing, a test that detects electrical activity in your brain, and neurofeedback for dementia therapy. It then settled on reminiscence therapy as the most compelling form of therapy that would benefit greatly from technology in an extremely user-friendly format.

"We were working with some new and untested ideas in this project, and trying to adapt rather sophisticated technology for mainstream users whose target group is actually supposed to be non-tech savvy. To overcome the former, we had a very intense period of EEG experiments and data analysis in order to develop algorithms for correlating EEG data to the application of reminiscence therapy," Yee said.

Plenty of time and manpower was also devoted to analysing, testing and tweaking the user experience design to make the process of using the system seamless for both the dementia sufferer and the caregiver, he explained. To promote the project, Ministry XR is releasing articles and publications, as well as working with dementia associations in Malaysia and Singapore to obtain more support in research development and user studies. Yee said the agency is also engaging with individuals and companies who may lend their expertise or monetary support in continuing this project to its full potential.

"Another avenue is promoting this work in XR and spatial computing circles, where we plan to publicise and talk about the project at spatial computing conferences and expos around the world," he added. According to him, spatial computing technology will drive Industry 5.0 and the agency hopes this project will "open the eyes" of mainstream society to this new shift in computing paradigm.

Meanwhile, CEO of Ministry XR, Ivan Khoo, added that the team's vision is to simplify lives by empowering all of humanity with computers, without having to think about how to us one. "We innovate towards a world where computers are invisible and perhaps then, when that day arrives, these misconceptions will finally be crushed," he told A+M.