Want to make creations as awesome as this one?

Transcript

Summary table

11-04-23

We need you!

Checklist tool

About the tool and how to use it

Discover the checklist tool to help you put the 8 principles into practice!

Discover the origin of the principles

We would like to improve this checklist. Do you have an opinion on any of the criteria or sub-criteria (a proposal for rewording, and sub-criteria that would be missing, ...)? Do not hesitate to contact us so that we can take it into account.We would also like to illustrate the application of these principles with concrete examples. You develop or have developed a solution, and you think that its development or the solution itself represents one or the other of the principles? Do not hesitate to let us know through the "Use cases" form. A library of use cases will be published soon.Because of the exhaustive nature of the work, you will find that all the criteria and sub-criteria form an impressive list of ideas and concepts to consider when developing or selecting a technological solution that meets these 8 principles. We don't want this to discourage anyone! We are therefore looking for ideas and examples to facilitate the use of this list. This could be specific situations where only certain criteria or sub-criteria are relevant to consider. It could be the creation of a pathway, allowing one principle to be considered at a time during the development of the solution. It could be ... Do you have an idea? Again, do not hesitate to contact us so that we can take it into account and why not develop it together!Contact: lara.vigneron@yuza.be

Our work will continue in many ways and we need your help!

Authors : Lara Vigneron (Yuza), Hilde Vandenhoudt (Licalab) Contact : lara.vigneron@yuza.be At the initiative of the King Baudouin Foundation and the Dr. Daniël De Coninck Fund, 8 guiding principles were born from a collective reflection on the role of technology in health in our societies. The current objective is to facilitate the implementation of these guiding principles to support the development of technologies that truly care for patients and serve citizens and society. A recurrent request from stakeholders (health professionals and technology developers) was to have a "checklist" type tool that would make the principles as concrete as possible and would allow to evaluate whether or not a technological solution meets the different principles and to what extent.This document presents a first version of this checklist. It should be seen as a working document, which will evolve over time according to the various feedbacks.This document is to be used as an inspirational guide, allowing to evaluate and evolve a technological solution. In no case should it be seen as a selection criteria grid allowing, for example through a scoring system, to "label" solutions.The work done so far has been as exhaustive as possible. The document is structured as follows: each principle is stated, then broken down into 3 more specific criteria. Each of these criteria is then broken down into a series of sub-criteria.

About the tool and how to use it

- Ensure that we understand the needs and desires of patients and citizens for health care and encourage the development of innovations and the use of data that meet those needs and desires, while maintaining a facilitative and supportive role.- Enable citizens to make their own autonomous and informed decisions based on their needs and wishes.

Criterion 1

Principle 1Citizen needs and autonomy

Criterion 3

Criterion2

C3. The technology solution helps citizens (or end users*) manage their own health and care needs and expectations (e.g., by easily and regularly accessing their data) empowers citizens (or end users*) to make their own decisions (e.g., by receiving enough information for the citizen to truly understand the ins and outs of the technology solution). *end users: people who ultimately use or are intended to use a product or service.

C1. Technology solution supports citizen health and quality of care.

C2. The development of the technology solution is based on a thorough understanding of the health and care needs and expectations of citizens (e.g., by conducting interviews, observations, pilot tests).

Sub-criteria

Criterion 1

C1.Technology solution supports citizen health and quality of care.

SC1. Effectiveness of the solution: The technological solution is effective in the management of patients, in the prevention, diagnosis or treatment of disease. It allows the patient to live a better life by improving their well-being and general health, reducing the symptoms of their disease, reducing the risk of complications related to the disease or treatment. SC2. Prevention education and awareness: The technology solution provides educational information and health tips to help patients understand risk factors, adopt a healthy lifestyle, and prevent disease. It educates patients on preventive practices, such as vaccination, early detection, stress management, etc.

MORE

Sub-criteria

Criterion 2

C2. The development of the technology solution is based on a thorough understanding of the health and care needs and expectations of citizens (e.g., by conducting interviews, observations, pilot tests).

SC1. In-depth research of patient needs and expectations: Prior to developing a technology solution, extensive research is conducted on patient needs and expectations, experiences and challenges encountered. The goal is to gather information that will help understand patient needs and expectations and determine the most important features for the technology solution. This research can be conducted through surveys, interviews, focus groups, observations, literature reviews, etc. A differentiation between patient-citizen needs and expectations is made. Needs and expectations may be validated by patient associations, official patient experience improvement bodies and/or patient committees. SC2. Involvement of health care professionals in needs and expectations: Healthcare professionals can also provide valuable information about patient needs and challenges. It is important to collaborate with healthcare professionals to ensure that the technology solution is compatible with existing care practices and that it is integrated into the workflows of healthcare professionals. SC3. Patient involvement in development: Patients are involved early in the development of the technology solution. This can be done through working groups, design workshops, testing, feedback on prototypes, etc. that will allow patients to provide feedback and ideas that will help shape the technology solution to meet their needs and expectations. For example, testing with patients can provide valuable information about the usability and effectiveness of the technology solution. Testing is conducted at various stages of development to ensure that the solution is being optimized for patients.

Continued ...

Sub-criteria

Criterion 2

C2. The development of the technology solution is based on a thorough understanding of the health and care needs and expectations of citizens (e.g., by conducting interviews, observations, pilot tests).

SC4. Post-launch patient engagement: After the technology solution is launched, it is important to track patient feedback and comments to ensure that it is meeting their needs. This can be done through surveys, online evaluations, in-app comments, etc. Patient feedback and comments can help identify areas for improvement in the technology solution. SC5. Validated and accessible patient engagement process: The process by which patients were engaged in understanding their needs and expectations is scientifically valid (in terms of representativeness, diversity, etc.) and accessible. SC6. Research findings on needs and expectations are accessible: The specific patient needs and expectations that the technology solution addresses are identified, and the information is accessible.

MORE

Sub-criteria

Criterion 3

C3. The technology solution helps citizens (or end users*) manage their own health and care needs and expectations (e.g., by easily and regularly accessing their data) empowers citizens (or end users*) to make their own decisions (e.g., by receiving enough information for the citizen to truly understand the ins and outs of the technology solution).*end users: people who ultimately use or are intended to use a product or service.

SC1. Ease of use of the solution: The technology solution is easy to use for the patient, so that they can easily access the information and tools they need to manage their disease. This includes intuitive interfaces and quick learning curve, familiar designs, etc., to reduce barriers to adoption. The solution can also be developed with so-called ambient technologies (immersed in the environment), requiring little or no interaction with the patient. SC2. Customization of the solution: The solution allows the patient to customize the functionalities according to his/her individual needs. For example, a diabetic patient should be able to adjust blood glucose thresholds in a blood glucose monitoring application based on their goals and preferences. The solution can be customizable based on the patient's skills in using technology solutions. It can also be adapted according to the patient's emotions, health status and state of mind. SC3. Ease of access to the solution: The solution is easily accessible, whether via a smartphone, tablet or computer, so that the patient can access it at any time and from anywhere. The solution also provides easy access to health information, including test results and lab reports.

C3. The technology solution helps citizens (or end users*) manage their own health and care needs and expectations (e.g., by easily and regularly accessing their data) empowers citizens (or end users*) to make their own decisions (e.g., by receiving enough information for the citizen to truly understand the ins and outs of the technology solution).*end users: people who ultimately use or are intended to use a product or service.

Continued ...

MORE

Sub-criteria

Criterion 3

SC4. Health tracking and monitoring: The solution is able to monitor the patient's symptoms and progress over time, and provide alerts or reminders when necessary. This may include reminders to take medications, reminders to perform tests or exams, or alerts for changes in symptoms. SC5. Improved coordination and communication between patients and healthcare professionals: The solution enables care coordination between the different healthcare professionals involved in the patient's treatment. The solution facilitates communication between healthcare professionals, as well as between healthcare professionals and patients, which enables regular monitoring of the patient's health status, improves the quality of care and can reduce patient stress. SC6. Integration for a complete user experience: The solution integrates with other systems or devices used by the patient, such as health monitoring devices, to provide a more consistent and complete user experience. More broadly, the solution integrates with the patient journey.

C3. The technology solution helps citizens (or end users*) manage their own health and care needs and expectations (e.g., by easily and regularly accessing their data) empowers citizens (or end users*) to make their own decisions (e.g., by receiving enough information for the citizen to truly understand the ins and outs of the technology solution).*end users: people who ultimately use or are intended to use a product or service.

Continued ...

Sub-criteria

Criterion 3

SC7. Education and information for empowerment: The solution provides information and educational resources to help the patient better understand their disease and best practices for managing it, while alerting to the risks of self-medication. The patient is also informed of the benefits of using the solution, and the needs and expectations it addresses. SC8. Integrated patient health goals: The solution provides tools to help the patient set health goals and track their progress. The solution can also include positive feedback to encourage the patient to take control of their health condition. SC9. Risk factor management support: The solution provides tools to help the patient manage risk factors related to their disease, such as nutrition and exercise. SC10. Patient choice to use the solution: Patients have the freedom to choose whether or not to use the solution, without impacting the quality of care and health monitoring they receive.

- Encourage ongoing collaboration among all stakeholders, including citizens, by creating a health technology ecosystem* (a space for exchange and meetings) open to the greatest number of people and organizations.- Promote an integrated ecosystem in which interoperability*, standardized protocols* and open source technologies* are self-evident in order to make innovations accessible to as many people as possible.* ecosystem: a system of living organisms (e.g., people and organizations) that interact with each other and with their environment.* interoperability: allows systems to exchange information with each other.* standardized protocols: clearly defined guidelines.* open source technology: technology whose source code is freely accessible.

Criterion 1

Principle 2Technology openness and integration

Criterion 3

Criterion 2

C3. The technology solution uses open source healthcare technology to facilitate its accessibility.

C1. The technology solution is developed in a way that fully supports and integrates ongoing end user and stakeholder* participation and collaboration to increase its relevance (e.g., through collaboration with patients, formal and informal caregivers, technology developers, insurers, etc.). *stakeholders: generally refers to all actors who have any interest in the development or implementation of the technology solution.

C2. The technology solution has a high level of interoperability and is integrated into the region's or country's existing healthcare infrastructure (e.g., using agreed-upon standardized protocols) to facilitate its adoption.

MORE

Sub-criteria

Criterion 1

C1. The technology solution is developed in a way that fully supports and integrates ongoing end-user and stakeholder* participation and collaboration to increase its relevance (e.g., through collaboration with patients, formal and informal caregivers, technology developers, insurers, etc.). *stakeholders: generally refers to all actors who have any interest in the development or implementation of the innovation.

SC1. Stakeholder involvement in development: The development of the technology solution involves the active participation of external stakeholders such as patients, health professionals, non-profit organizations, companies, or universities. In this collaboration, patients are considered experts in their lives and health conditions. SC2. Exchange of ideas and knowledge among stakeholders: Stakeholders have the opportunity to exchange ideas and knowledge in the development of the technology solution. Meetings are held to gather feedback and suggestions, working groups are created to solve problems and develop ideas together. Online platforms or tools are eventually used to allow stakeholders to share ideas, feedback, and knowledge. These exchange platforms may include online forums, social networks, collaborative websites, document sharing tools, etc. SC3. Transparency and communication about the solution development process: Stakeholders are informed about the technology solution development process. Information about goals, progress, challenges, outcomes, and costs are communicated in a transparent manner.

Continued ...

Sub-criteria

Criterion 1

C1. The technology solution is developed in a way that fully supports and integrates ongoing end-user and stakeholder* participation and collaboration to increase its relevance (e.g., through collaboration with patients, formal and informal caregivers, technology developers, insurers, etc.). *stakeholders: generally refers to all actors who have any interest in the development or implementation of the technology solution.

SC4. Clear definition of roles and responsibilities in development: Stakeholders are informed of their role in the development of the solution. Stakeholders have the opportunity to contribute to the development of the technology solution and share in the benefits. For example, the rules for sharing intellectual property are defined from the beginning. SC5. Social impact: The technology solution is developed with the objective of meeting the needs of patients, improving healthcare and having a positive impact on society in general.

Sub-criteria

Criterion 2

C2. The technology solution has a high level of interoperability and is integrated into the region's or country's existing healthcare infrastructure (e.g., using agreed-upon standardized protocols) to facilitate its adoption.

SC1. Following standards and interoperability protocols: The technology solution is designed according to recognized healthcare standards and interoperability protocols such as HL7 FHIR (Fast Healthcare Interoperability Resources), DICOM (Digital Imaging and Communications in Medicine) or IHE (Integrating the Healthcare Enterprise). SC2. Ability to share data: The technology solution is capable of sharing data transparently with other healthcare systems, using interoperability standards to ensure the quality and consistency of the data exchanged. SC3. Security of data exchange: The technology solution ensures the security and confidentiality of the data exchanged, complying with privacy standards and security requirements in the healthcare sector. SC4. Ease of integration: The technological solution is easy to use and to integrate with other health systems, minimizing technical and organizational barriers for data exchange. In particular, the solution interfaces with the "official" technological solutions of the health care services, such as the Walloon Health Network, Abrumet, Vitalink, mijngezondheid.be, masanté.be, etc., but also with the portal of hospitals, etc. It interfaces with other apps/SDs so that the data can be used by everyone in the care pathway. SC5. Documentation and technical support for interoperability: The technology solution is well documented and comes with technical support to help users understand and integrate the interoperability features of the solution. The solution also ensures interoperability with older/existing and new systems.

Sub-criteria

Criterion 3

C3.The technology solution uses open source healthcare technology to facilitate its accessibility.

SC1. Source code availability: The technological solution is based on open source software whose source code is freely available and can be modified and distributed. SC2. Open standards: The technology solution is designed using open standards to ensure interoperability with other health systems. SC3. Open source license: The technology solution is distributed under an open source license approved by the Open Source Initiative, such as the Apache License or the GNU GPL. SC4. Developer community support: The technology solution is developed transparently and is supported by an active community of open source developers. While respecting its confidentiality, the data generated is open for benchmarking. SC5. Documentation and technical support for open source: The technology solution is accompanied by clear documentation and technical support to help users understand and use the functionality of the solution.

- Provide and give easy access to transparent, honest, reliable, and comprehensive information about health and care technology solutions.- Ensure that the usefulness, applicability, advantages and disadvantages of solutions are described in a way that is understandable to citizens.

Criterion 1

Principle 3Transparency and clarity of information

Criterion 3

Criterion 2

C3. End users have easy access to all information about the technology solution (e.g., by clearly indicating where to look for information).

C1. End users receive honest, reliable, transparent, and complete information about the technology solution in terms of its usefulness, applicability, advantages, and disadvantages (e.g., by stating whether an independent source has verified the information about the solution).

C2. End users receive information in a way that is easy to understand (e.g., by not using jargon, by providing access to a list of terms or a glossary, etc.).

MORE

Sub-criteria

Criterion 1

C1. End users receive honest, reliable, transparent, and complete information about the technology solution in terms of its usefulness, applicability, advantages, and disadvantages (e.g., by stating whether an independent source has verified the information about the solution).

SC1. Information on the solution's functionalities: The user is informed of the solution's functionalities, including the advantages and limitations of each functionality, the risks involved in using the solution. The user is also informed of the needs that the solution addresses and the targeted user profiles. SC2. Information on security and confidentiality: The user is informed of the security and confidentiality measures put in place to protect his/her health data. They are also informed about the use of their data, the possible sharing with third parties, and the existence of a consent in relation to the sharing of data. SC3. Information on technical requirements: The user is informed of the technical requirements necessary to use the solution, such as hardware and software requirements. SC4. Cost information: The user is informed of the costs associated with using the solution, including initial fees, subscription fees, maintenance fees, and any additional costs. SC5. Information on assessments and certifications: The user is informed about assessments and certifications obtained by the solution, such as ISO 13485 certifications, as well as assessments performed by third parties (such as mHealthBelgium or other quality labels), such as safety and interoperability

Continued ...

Sub-criteria

Criterion 1

C1. End users receive honest, reliable, transparent, and complete information about the technology solution in terms of its usefulness, applicability, advantages, and disadvantages (e.g., by stating whether an independent source has verified the information about the solution).

SC6. Information about the provider: The user is informed about the company or organization providing the solution, including its experience, reputation and track record in compliance and quality, its sources of revenue, especially if the solution is free to use. It is also informed of the expertise of the content authors, statements of interest, key sources, and bibliographic references, etc. SC7. Information updates: Information is regularly updated to reflect changes in technology, new research or new quality and safety standards.

MORE

Sub-criteria

Criterion 2

C2. End users receive information in a way that is easy to understand (e.g., by not using jargon, by providing access to a list of terms or a glossary, etc.).

SC1. Simple, accessible language: Information is presented in simple, accessible language that the general public can understand, without using complex technical or medical terms. Standardized tools/formats are preferred (e.g., medication inserts). A glossary or list of terms is provided if necessary to facilitate understanding of the information given, as well as additional content in the form of questions and answers. SC2. Clear presentation of information: Information is presented in a clear and concise manner, using headings and subheadings to facilitate reading and understanding. Oral information is not delivered too quickly. Illustrations and graphics can help clarify complex information or provide a visual representation of data. SC3. Use of interactive media: Interactive media such as videos and animations can help explain concepts more effectively than text alone. SC4. Repetition of information: Key information is repeated several times throughout the document or presentation to aid recall and understanding. SC5. Adaptation of information to the user's needs: Information is adapted to the specific needs of the user, considering his or her medical history, level of understanding and preferences. Written text can be enlarged, color contrast can be adjusted.

Continued ...

Sub-criteria

Criterion 2

C2. End users receive information in a way that is easy to understand (e.g., by not using jargon, by providing access to a list of terms or a glossary, etc.).

SC6. Relevance of Information: Information provided is relevant to the user's needs, including medical condition, preferences, and medical history. SC7. User feedback on information: The information content has been developed and tested with users, for example through patient associations, consumer associations, or an organization such as 'Gezondheid en wetenschap' (Health and Science) to improve the quality, relevance and accessibility of the information and to ensure that it is easily understandable for all users.

Sub-criteria

Criterion 3

C3. End users have easy access to all information about the technology solution (e.g., by clearly indicating where to look for information).

SC1. Various communication channels: Several communication modalities/channels are available to access information through the solution itself, a website, a dedicated online platform or an information portal accessible at any time, as well as through brochures, posters for waiting rooms, etc. Communication and education campaigns are implemented for patients and professionals. SC2. Interactive help for accessing information: The solution provides interactive help for accessing information through a helpdesk, a chatbot, and possibly by proposing a referral person, etc. SC3. Education and training on the solution: The solution provides information through training for patients and professionals (general practitioners and specialists, pharmacists, nurses, etc.) as well as any relevant and trusted person who can play the role of intermediary and companion. The proposed training programs enable the use of the solution and understanding of its benefits, and eventually enable professionals to train their patients themselves. SC4. Patient choice to receive information: The technology solution allows the user, if he or she wishes, to not receive information.

- Strengthen citizens' and organizations' trust in the use of data and the development of technological solutions that use their data by ensuring that they own their data.- Help citizens share their data securely and use it as a lever for their personal well-being and for the public good.

Criterion 1

Principle 4Ownership and security of personal data

Criterion 3

Criterion 2

C3. Citizens share their data for their personal well-being and the public interest (e.g., by sharing their data to support better research).

C1. Citizens are informed about data ownership, data access and data use, with the ability to access and change consent at any time.

C2. Citizen data are collected, stored, and shared securely between end users and stakeholders, both of whom are notified of any security incidents.

Sub-criteria

Criterion 1

C1. Citizens are informed about data ownership, data access and data use, with the ability to access and change consent at any time.

SC1. Transparency and communication about data ownership and management: The user is informed in a clear and concise manner about data ownership, data access, and data use. The user is transparently informed about the sharing of their personal data, including the purpose and duration of the sharing. Commercial use of data is left to the patient's choice. SC2. Revocable consent: The user must give informed consent before their data is collected, stored, used, or shared by providing an explicit option to withhold consent. The user has the ability to change their consent at any time by providing an easy-to-use option to revoke consent. SC3. Privacy policy: The technology solution provides a comprehensive privacy policy, detailing how the user's data will be collected, stored and used, as well as the measures taken to protect the data. The sharing of personal data complies with applicable legal standards, including data protection and privacy.

Sub-criteria

Criterion 2

C2. Citizen data are collected, stored, and shared securely between end users and stakeholders, both of whom are notified of any security incidents.

SC1. Access to his or her data: The user has the ability to access their data at any time, providing an easy-to-use platform for viewing and editing data. SC2. Data portability: The user has the ability to retrieve their data and transfer it to another platform or system if necessary. SC3. Data traceability: The user has the ability to track the use of their personal data and withdraw their consent at any time. SC4. Data security and confidentiality: The technological solution guarantees the security of the user's data, in particular by using state-of-the-art security protocols to protect the data against any abusive or unauthorized use. In practice, this includes implementing user/data authentication to ensure data integrity and authenticity, and secure data storage on the device, on one or more remote servers. Data breaches, confidentiality violations and security incidents are reported transparently.

Sub-criteria

Criterion 3

C3. Citizens share their data for their personal well-being and the public interest (e.g., by sharing their data to support better research).

SC1. Relevance of data collected: Data are collected in a relevant manner (relevance of information to the context of use), while minimizing the amount of data collected (no collection of unnecessary data) for the personal well-being of the patient. SC2. Legitimate purpose of data sharing: The sharing of personal data is clearly defined and limited to a legitimate, public interest purpose, such as scientific research or public health. SC3. Transparency and communication about data sharing: Users are transparently informed about the sharing of their personal data, including the purpose and duration of the sharing. The solution encourages patients to share their data for the public good by having a communication about it, providing understandable information, and showing the (reciprocal) benefits as well as the disadvantages of such sharing (giving examples of outcomes). SC4. Data anonymization: The solution enables data sharing for the public good through anonymization, pseudonymization, etc., of collected data to ensure user privacy.

- Improve technological literacy*, health literacy* and participation for all citizens. Commit to lifelong learning for all.- Engage everyone, including the vulnerable and disadvantaged. Innovation must focus on bridging digital and health divides, not contributing to them.*literacy: the ability to find, understand and use information and services. This can include health and digital information and services.

Criterion 1

Principle 5Education and inclusion of citizens

Criterion 3

Criterion 2

C3. The technological solution is accessible to all citizens, and particularly benefits vulnerable and disadvantaged citizens (e.g., regardless of socio-economic, cultural, psychosocial, physical condition).

C1. The implementation of the technology solution improves the ability of citizens to better use the technology information they receive (e.g., by providing technical information in an educational manner or by offering assistance).

C2. The implementation of the technology solution improves the ability of citizens to better use the health information they receive (e.g., by increasing their awareness of the determinants of health).

MORE

Sub-criteria

Criterion 1

C1. The implementation of the technology solution improves the ability of citizens to better use the technology information they receive (e.g., by providing technical information in an educational manner or by offering assistance).

SC1. Technology-oriented educational content: The solution provides informative content delivered in an educational manner such as user guides, explanatory videos, visuals, and multimedia to help users understand how to use a technology solution by showing them how to perform specific actions. Videos, infographics, images, animations, and graphics can also help explain difficult concepts. SC2. Focus on personal data management: The solution provides users with informative content delivered in an educational manner to educate users on the importance of managing their personal data, the privacy rules that apply, and the use of consent for sharing their data. SC3. Interactive help for technology use: The solution offers online and offline technical support (through a helpdesk, chatbot, etc.) that is understandable to the user to solve problems that may arise when using the technology. SC4. Real-time technology feedback: The solution provides real-time feedback (real-time comments, such as error messages, notifications, and status updates) to help users understand what is happening when they interact with a technology solution. Error messages and notifications can also help users avoid errors and understand how to properly use the technology.

Continued ...

Sub-criteria

Criterion 1

C1. The implementation of the technology solution improves the ability of citizens to better use the technology information they receive (e.g., by providing technical information in an educational manner or by offering assistance).

SC5. Support forums: The solution provides or references support forums to help promote technology literacy by providing useful information. SC6. Technology education and training: Users are provided with a training program that helps them understand how to use the solution by showing them how to perform specific actions.

Sub-criteria

Criterion 2

C2. The implementation of the technology solution improves the ability of citizens to better use the health information they receive (e.g., by increasing their awareness of the determinants of health).

SC1. Health-oriented educational content: The solution provides informative content delivered in an educational manner such as visuals and multimedia to help users better understand health and prevention information. Images, videos and graphics can help users visualize health concepts in a clearer and more concise way. SC2. Focus on stigma issues: The solution raises awareness of stigma issues related to mental health, chronic diseases, etc., by educating and informing users about causes, symptoms, and treatments to reduce misunderstanding and prejudice towards these conditions. SC3. Real-time health feedback: The solution provides real-time feedback (comments, notifications) to help users better understand the consequences of their health choices. SC4. Patient forums: The solution offers or references patient and citizen forums to help them promote health literacy by providing useful information. SC5. Health education and training: Users are provided with a training program that enables them to use the solution and understand its health benefits.

Sub-criteria

Criterion 3

C3. The technological solution is accessible to all citizens, and particularly benefits vulnerable and disadvantaged citizens (e.g., regardless of socio-economic, cultural, psychosocial, physical condition).

SC1. Affordability: The solution is affordable to all citizens, regardless of their socio-economic status. The costs associated with using the solution are transparent and predictable. SC2. Geographic accessibility: The solution is geographically accessible to all citizens, including those living in rural or remote areas. Digital solutions can help address this issue. SC3. Linguistic and cultural accessibility: The solution is accessible to all citizens, regardless of their native language or culture. The solution's interfaces are available in local languages and the solution takes into account cultural differences to be relevant and acceptable. SC4. Cognitive or physical disabilities addressed: The solution is easy to use for all citizens, including those with cognitive or physical difficulties whether related to illness, advancing age, or any other reason. The interfaces, contents, and the way they are delivered (text, audio-visual aids, etc.) are simple, intuitive and inclusive to enable all users to use them. This includes, for example, hearing, vision, motor, learning, reading difficulties, etc. SC5. Non-stigmatizing language: The solution uses non-stigmatizing language to describe diseases or health problems. The choice of words and phrases is respectful and inclusive of all users.

- Encourage citizens and stakeholders to actively participate in the emergence and deployment of the health technology ecosystem.- Develop participatory and adaptive governance* at the ecosystem and innovation levels.*governance: a set of measures, rules, decision-making, information and monitoring bodies that ensure the proper functioning and control of an organization or a group of people.

Criterion 1

Principle 6Participation and shared governance

Criterion 3

Criterion 2

C3. Governance has an adaptive process for updating its components and the roles of individuals and organizations through collective learning with new data, experience, evidence, and growing expertise.

C1. End users and stakeholder representatives (i.e., education and research, civil society, government, business) are actively involved in the governance of the ecosystem or technology solution (e.g., through an oversight group or steering committee).

C2. Governance is monitored and evaluated on a regular basis (e.g., by assessing the interaction and distribution of power, adherence to values, composition of governance, etc.).

MORE

Sub-criteria

Criterion 1

C1. End users and stakeholder representatives (i.e., education and research, civil society, government, business) are actively involved in the governance of the ecosystem or technology solution (e.g., through an oversight group or steering committee).

SC1. Respect for diversity: Users and stakeholders represent a wide range of needs, experiences, and issues. Project or ecosystem governance takes this diversity into account and ensures that all voices are heard and considered. SC2. Stakeholder involvement in governance: Stakeholders, including patients, healthcare professionals, healthcare administrators, and technology solution developers, are involved in governance from the beginning of the technology solution's development, from its design and planning phase. Stakeholders are informed of the project objectives and the processes that will be used to achieve those objectives. SC3. Active, continuous, and collaborative participation in governance: Users and stakeholders participate actively and in a continuous and collaborative manner in the governance related to the development and implementation of the technology solution or ecosystem. They are involved in making important decisions and in defining objectives and priorities. SC4. Transparency and communication of governance: Users and stakeholders are informed in a transparent and regular manner about the progress of the solution or ecosystem, the challenges encountered, and the decisions taken. Open and regular communication with users and stakeholders is essential to build user trust and commitment to the project or ecosystem.

Continued ...

Sub-criteria

Criterion 1

C1. End users and stakeholder representatives (i.e., education and research, civil society, government, business) are actively involved in the governance of the ecosystem or technology solution (e.g., through an oversight group or steering committee).

SC5. Participant recognition: Users and various stakeholders are recognized for their active participation in the governance of the project or ecosystem. This may include formal recognition of their contribution, participation in events or conferences, or compensation for their time and expertise.

Sub-criteria

Criterion 2

C2. Governance is monitored and evaluated on a regular basis (e.g., by assessing the interaction and distribution of power, adherence to values, composition of governance, etc.).

SC1. Clear definition of governance roles and responsibilities: The roles and responsibilities of each stakeholder in the governance of the project are clearly defined and transparently communicated. SC2. Regular evaluation of governance: The solution or ecosystem is evaluated regularly to measure progress and determine if objectives are being met effectively. The evaluation is based on hard, objective data and is conducted by independent stakeholders. The results of the assessment are analyzed to understand the causes of deviations from the objectives and to determine corrective actions. SC3. Transparency and communication of governance assessment results: The results of the assessment are communicated in a transparent manner to all stakeholders involved in the project. Stakeholders should be encouraged to provide feedback and participate in the implementation of corrective actions. SC4. Continuous improvement of governance: The results of the evaluation are used to improve the governance of the solution or ecosystem and to determine what adjustments are needed to ensure that the objectives are met effectively. Project governance is flexible to allow for adjustments as the solution environment or ecosystem changes.

Sub-criteria

Criterion 3

C3. Governance has an adaptive process for updating its components and the roles of individuals and organizations through collective learning with new data, experience, evidence, and growing expertise.

SC1. Continuous governance learning: Governance members continuously train and acquire new knowledge through exchanges, publications, etc. SC2. Knowledge sharing and management: Governance members share their knowledge and skills to help other members learn and develop. This knowledge is collected, and any new member can access, understand and integrate it into their practice. SC3. Experimentation and knowledge capitalization: Governance members focus on experimentation to improve the way they operate by testing new roles, new ways of making decisions, etc., evaluating the results and capitalizing on the knowledge gained. SC4. Governance adaptability: Governance members are able to adapt to new challenges and situations, including societal, social, and technological changes, by learning from experience and making changes to improve the work of governance.

- Continuously evaluate the development and implementation of innovation and the use of data. Monitoring should include content, security, information transparency, traceability, utility, effectiveness, interoperability, inclusiveness, and autonomy. Evaluation is based on feedback and scientific evidence. Lessons learned must be matched with scientific evidence.- Introduce quality labels* and disseminate evaluation results.*quality label: it certifies that a product or service has been designed according to certain quality criteria or standards.

Criterion 1

Principle 7Evaluation and quality of technology solutions

Criterion 3

Criterion 2

C3. The results of the evaluation of the technology solution are made accessible to all stakeholders, for example through a quality label.

C1. The development and implementation of the technology solution and the use of the data are monitored and evaluated regularly so that flexible adjustments can be made if necessary. Evaluation is based on lessons learned and scientific evidence.

C2. Monitoring includes content, security, information transparency, traceability, efficiency, interoperability, inclusiveness, autonomy, and perceived and actual impact on end users.

MORE

Sub-criteria

Criterion 1

C1. The development and implementation of the technology solution and the use of the data are monitored and evaluated regularly so that flexible adjustments can be made if necessary. Evaluation is based on lessons learned and scientific evidence.

The health technology solution assessment process should include the following elements: SC1. Clear evaluation objectives: The objectives of the evaluation are clearly defined and understood by all stakeholders involved in the evaluation process. SC2. Defined and measurable evaluation criteria: Evaluation criteria are defined and measurable for each health technology solution. The evaluation criteria are aligned with the objectives of the evaluation. SC3. Quality data collection: Reliable and valid real-world data are collected for each evaluation criterion. Data are collected from multiple sources, such as patients, healthcare professionals, and electronic health records. SC4. Quality data analysis: The collected data are analyzed to determine the strengths and weaknesses of each health technology solution. Data analysis is conducted in a scientific (objective and unbiased) manner by independent experts. For example, data are collected over sufficiently long periods of time, control groups must be set up, etc.

C1. The development and implementation of the technology solution and the use of the data are monitored and evaluated regularly so that flexible adjustments can be made if necessary. Evaluation is based on lessons learned and scientific evidence.

Continued ...

MORE

Sub-criteria

Criterion 1

SC5. Stakeholder involvement in the evaluation: Stakeholders, including patients, health professionals, health care administrators, and technology developers, are involved in the evaluation process to ensure that their perspectives are taken into account. Ways to collect user feedback include surveys (interviews or online questionnaires), focus groups (e.g., patient committees), observations, usage data, comments left by users in the solution (free or through questionnaires), online evaluations (on websites, forums, etc.). SC6. Ongoing evaluation of solutions: Evaluation of health technology solutions is ongoing to ensure that the results achieved are sustainable and continue to meet the needs of patients and healthcare professionals. SC7. Lifecycle considerations: The technology solution is evaluated throughout its lifecycle, including design, implementation, maintenance, and end-of-life. SC8. Continuous improvement of the technology solution: The results of the evaluation are used to improve the technology solution and to determine what adjustments are needed to ensure that it effectively meets its objectives. In particular, the evaluation process in place allows for a timely response to any anomalies or risks detected.

C1. The development and implementation of the technology solution and the use of the data are monitored and evaluated regularly so that flexible adjustments can be made if necessary. Evaluation is based on lessons learned and scientific evidence.

Continued ...

Sub-criteria

Criterion 1

SC9. Completeness of the evaluation: Regardless of the number of organizations and individuals involved in the evaluation of the technology solution, all of the information and results of the evaluation are brought together to provide a comprehensive and complete understanding of the evaluation. SC10. Transparency and communication of the solution evaluation process: The evaluation process, including its objectives, criteria, analysis methods, evaluators, etc., is transparent and communicated to all stakeholders.

MORE

Sub-criteria

Criterion 2

C2. Monitoring includes content, security, information transparency, traceability, efficiency, interoperability, inclusiveness, autonomy, and perceived and actual impact on end users.

The solution is evaluated on the following criteria: SC1. Health impact for the patient (Principle 1): The solution enables the patient to live a better life by reducing the symptoms of their disease, improving their overall well-being, or reducing the risk of disease or treatment-related complications. SC2. Impact for healthcare professionals (Principle 1): The solution enables healthcare professionals to deliver better or more efficient care (more accurate and timely diagnosis, more appropriate treatment, etc.), while minimizing potential negative impacts to them. The solution can affect workload, ease of collaboration between professionals, patient-professional relationship, etc. SC3. Needs, support and autonomy (Principle 1): The solution meets the patient's needs and expectations, helps them manage their health condition, and promotes their autonomy. SC4. Openness and integration (Principle 2): Users and stakeholders actively participate in the development and involvement of the solution. The solution is interoperable and open source and is integrated into the healthcare and patient pathways. SC5. Transparency and clarity of information (Principle 3): Information provided about and by the solution is honest, reliable, transparent, and complete, easily understood and accessible by all.

C2. Monitoring includes content, security, information transparency, traceability, efficiency, interoperability, inclusiveness, autonomy, and perceived and actual impact on end users.

Continued ...

Sub-criteria

Criterion 2

SC6. Ownership and Security of Personal Data (Principle 4): Citizens are informed about data ownership and have the ability to manage access and sharing of their data. Data is collected, stored, and shared in a secure manner. For the well-being of patients but potentially also for the public good. SC7. Technology literacy, health literacy and inclusion (Principle 5): The solution improves the ability of citizens to better use technology and health information. It is accessible to all. SC8. Shared Governance (Principle 6): The governance of the solution is open to patients and stakeholders and is shared. Governance members act as a learning community. SC9. Standards (Principle 7): The solution follows current standards related to medical certification if applicable, quality standards (ISO, etc.), privacy rules (GDPR, etc.), etc. SC10. Health, sustainability and ethical issues (Principle 8): The solution responds to major societal issues, in particular responds to health issues, sustainable development objectives relevant to the impact it may have

MORE

Sub-criteria

Criterion 3

C3. The results of the evaluation of the technology solution are made accessible to all stakeholders, for example through a quality label.

SC1. Freedom and ease of access to evaluation results: Evaluation results and findings, as well as the physical or virtual locations where they are available, are communicated to all stakeholders and everyone has the freedom to access them. These locations take into account potential accessibility challenges related to financial, geographic, cultural, language, cognitive and physical disabilities. SC2. Following standards and guidelines for health solutions: The solution follows clear and recognized standards and guidelines. These standards and guidelines can help clarify expectations and requirements for health technologies, which can increase user confidence. These standards or guidelines can exist for example in the form of an independent quality label, or rules qualifying the reimbursement of the solution by the healthcare system (e.g. mhealthBelgium). SC3. Network and community support: The solution is supported by networks and communities that can help promote the use of positively evaluated health technologies by providing a physical or virtual space to share knowledge, experiences, and best practices. Communities can also foster collaboration among users, health professionals, and policy makers, which can enhance technology adoption and use. SC4. Integration into education and training: The solution is offered in recognized training and education programs that can help promote the use of positively evaluated health technologies by providing users with the skills and knowledge to use them effectively. These programs can also help address user concerns, which can build user confidence and satisfaction with the technology.

Continued ...

Sub-criteria

Criterion 3

C3. The results of the evaluation of the technology solution are made accessible to all stakeholders, for example through a quality label.

SC5. Promoting the solution: The solution benefits from technology promotion campaigns that can help promote the use of positively evaluated health technologies by increasing awareness and adoption of the technology. Promotional strategies may include awareness campaigns, online or in-person demonstrations, publications, presentations at conferences and other events, and use of social media to reach potential users.

- Monitor the actions of the health technology ecosystem and evaluate whether they are consistent with the goals set for health and care within a broader framework of prevention, ethics, and sustainability. If necessary to strengthen this coherence, adjust policies in a flexible but decisive manner, based on data, experience, evidence, and growing expertise.- Integrate specific sustainable development goals* and appropriate ethical principles* (e.g., human rights) into the development of innovations.*sustainable Development Goals (SDGs): they define 17 priorities for socially equitable, environmentally safe, economically prosperous, inclusive and predictable development.*ethical principles: ethical principles commonly associated with health include the principles of autonomy, beneficence, non-maleficence and justice.

Criterion 1

Principle 8Consistency with the political and societal framework

Criterion 3

Criterion 2

C3. The technological solution incorporates specific ethical principles (e.g. avoiding infringement of fundamental moral or cultural values).

C1. Ecosystem actions, the technology solution itself, and policies are monitored and adapted to ensure that they are beneficial to society (i.e., that the likely benefits substantially outweigh the reasonably foreseeable risks and harms) and are consistent with identified health goals (including health prevention).

C2. The technology solution incorporates specific sustainable development goals (e.g., minimizing its environmental footprint during development or use).

Sub-criteria

Criterion 1

C1. Ecosystem actions, the technology solution itself, and policies are monitored and adapted to ensure that they are beneficial to society (i.e., that the likely benefits substantially outweigh the reasonably foreseeable risks and harms) and are consistent with identified health goals (including health prevention).

SC1. Response to social and societal challenges: The solution responds to the needs and challenges faced by the healthcare system, such as the lack of qualified healthcare professionals, the growing demand for healthcare due to the aging population and the burden of chronic diseases, the response to epidemics and pandemics, etc. SC2. Solution impact evaluation: The impact of the technology solution on the healthcare system is evaluated regularly to identify areas for improvement and ensure that the technology solution continues to meet the needs of the healthcare system. SC3. Cost-effectiveness of the solution: The technology solution is cost-effective and economically viable. It provides value for the costs incurred in its development, deployment, and maintenance. It reduces health care costs, for example, by reducing the number of hospital visits, enabling more efficient use of health care resources (time savings, better coordination, etc.), etc. SC4. Preventive interventions: The technology solution enables the implementation of preventive interventions, such as vaccination reminders, screening tests, prevention advice, recommendations for a healthy lifestyle, etc. It is able to evaluate the effectiveness of the preventive interventions implemented, in order to adapt the interventions according to the needs.

MORE

Sub-criteria

Criterion 2

C2. The technology solution incorporates specific sustainable development goals (e.g., minimizing its environmental footprint during development or use).

The development and use of health technology solutions can impact several of the Sustainable Development Goals (SDGs). Some are already addressed through other guiding principles, including: • SDG 3: Good health and well-being: Health technology solutions can improve access to quality health care, support disease prevention, improve diagnosis and treatment of disease, and improve mental health. Health technology solutions can also improve the management of epidemics and pandemics (Principle 1 and Principle 8). • SDG 9: Industry, innovation, and infrastructure: Health technology solutions can contribute to innovation in healthcare, modernization of health infrastructure and improved access to healthcare (Principle 6). • SDG 10: Reducing inequalities: Health technology solutions can help reduce health inequalities by enabling equitable access to healthcare and improving the quality of care for all, regardless of location or socio-economic status (Principle 5). • SDG 11: Sustainable cities and communities: Health technology solutions can improve access to healthcare in urban areas, especially in densely populated urban areas (Principle 5). • SDG 16: Peace, justice and effective institutions: Health technology solutions can help promote transparency, accountability and citizen participation in healthcare management and improve the quality of health services (Principle 6). .

Continued ...

MORE

Sub-criteria

Criterion 2

C2. The technology solution incorporates specific sustainable development goals (e.g., minimizing its environmental footprint during development or use).

• SDG 17: Partnerships for achieving the goals: Health technology solutions can stimulate collaboration among health professionals, patients, governments, nongovernmental organizations, and businesses to improve health and well-being (Principle 2 and Principle 6). In addition, the development and use of health technology solutions can impact: • SDG 12: Sustainable consumption and production: Health technology solutions can help reduce the ecological footprint of healthcare by reducing medical waste, promoting sustainable healthcare practices, and reducing travel for patients and healthcare professionals. Here are some criteria that can be considered to say that a health technology solution minimizes its environmental footprint: SC1. Efficient use of natural resources: The technology solution is designed to minimize the use of natural resources such as energy, water, and materials. SC2. Low energy consumption: The technology solution consumes the least amount of energy possible, using energy-efficient technologies or optimizing processes to avoid energy waste. SC3. Waste minimization: The technology solution is designed to minimize waste generation and to enable recycling and reuse of materials.

Continued ...

Sub-criteria

Criterion 2

C2. The technology solution incorporates specific sustainable development goals (e.g., minimizing its environmental footprint during development or use).

SC4. Emissions reduction: The technology solution is designed to minimize emissions of greenhouse gases and other air pollutants. SC5. Life cycle evaluation: The technology solution is evaluated over its entire life cycle, from production to disposal, to identify opportunities to reduce environmental impact. SC6. Environmental certification: The technology solution is certified to environmental standards, such as ISO 14001, to ensure compliance with international environmental standards. SC7. Environmental awareness: The technology solution promotes environmental awareness and engagement among users by providing information on environmental impacts and encouraging environmentally responsible behavior.

MORE

Sub-criteria

Criterion 3

C3. The technological solution incorporates specific ethical principles (e.g. avoiding infringement of fundamental moral or cultural values).

SC1. Equity and solidarity: The benefits and risks of using the technology solution are distributed equitably among different population groups, and the data collected are not used to discriminate against individuals or population groups. The solution is particularly attentive to the needs of marginalized or vulnerable groups in society and ensures that they also benefit from access to the technology. SC2. Frugality: The solution implies the lowest possible production and maintenance requirements, focuses on basic functionalities that can meet the needs of the largest number of users, making the solution simple and without superfluous additions, efficient and robust to take into account different contexts of use (difficult climatic conditions, ease of transport, etc.). SC3. Explicability: The functioning of the technological solution is clearly understandable and explainable to users and health professionals. This can be done by using transparent algorithms and providing detailed explanations of the processes used to obtain the results. This allows healthcare professionals to make informed decisions and patients to understand the reasons behind recommendations and proposed treatments. SC4. Justifiability: The decisions made by the technology solution must be explainable and justifiable in ethical and legal terms. This implies that the technology must be designed to allow for greater transparency and accountability by providing clear explanations of algorithmic choices and resulting decisions. Users of the technology must be able to understand why a decision was made and must have the ability to challenge that decision if necessary.

Continued ...

Sub-criteria

Criterion 3

C3. The technological solution incorporates specific ethical principles (e.g. avoiding infringement of fundamental moral or cultural values).

SC5. Safe and healthy working conditions: The working conditions of the healthcare professionals who use the technology solution, as well as the working conditions of the designers, manufacturers, and distributors of the solution, are safe and healthy (safety, proper working hours, fair pay, respect for privacy, etc.), and free from discrimination (based on age, gender, race, religion, sexual orientation, etc.). SC6. Time savings: The technology solution saves time for patients and healthcare professionals, e.g., by reducing the time needed to obtain test results, enabling remote monitoring, automating certain administrative tasks, etc.

MORE

Summary table

MORE

Summary table

MORE

Summary table

MORE

Summary table

Summary table