Technical Reports

PITHIA-NRF provides access to research facilities for observations of the upper atmosphere (ionosphere-thermosphere-plasmasphere), through the Trans-National Access (TNA) programme, organised with twelve nodes providing access to key experimental and data processing facilities, geographically distributed over Europe. TNA is coordinated by the PITHIA Access Bureau (PAB), represented by staff from the twelve nodes, and will ensure that project selection criteria are applied and that a correct balance of projects is achieved, for example. A support centre which will be a connection point between users and PAB. 

There will be a call twice per year where potential users can propose projects by filling in an online application form, describing their scientific idea, and they can request either a physical access (one-week visit) or a one-month remote access. The calls will be advertised and information about the different PITHIA-NRF nodes will be communicated. Printed materials (posters, pamphlets) will be available at conferences. 

Access is granted or denied based on the feasibility, excellence and technical merit of the proposal. A selection procedure is required also due to limited resources. This process must contain an eligibility check, a feasibility check, and a review. The access provider provides hands-on support during the access. For the rest of the time of the six-month project remote support is provided. When a project has been concluded, the user is required to submit an executive report of the outcome of the project and an evaluation of the TNA experience.

 

Research agencies and laboratories in several EU member states operate a large number of excellent facilities for the observation of the Earth’s Ionosphere, Thermosphere and Plasmasphere. However, their operation is mainly uncoordinated using different standards and different policies for access and exploitation which are primarily tuned to national interests and priorities. Due to the uncoordinated operation of the facilities, researchers in Europe and worldwide cannot optimally exploit their full potential without spending substantial resources and time, despite the significant investment made mainly through national and regional funds.

Users and stakeholders in the broad scientific domain of the upper atmosphere and near-Earth space research, wish to benefit from an easy and well-organized access to data, models and experimental facilities and to e-science tools that will enable research advances and innovation. To meet this challenge PITHIA-NRF establishes a unique European Infrastructure that provides open access to relevant e-services and trans-national access to experimental facilities. PITHIA-NRF has the ambition to become a European hub that will act as facilitator for coordinated observations from ground and space, for provision of data, processing tools and modelling advances and for software and data-products standardization. PITHIA-NRF will also offer advice on the transitioning of models to operations providing e-science supporting tools so that models can reach the desired accuracy and standards.

 

PITHIA-NRF provides access to research facilities for observations of the upper atmosphere (ionosphere-thermosphere-plasmasphere) through the Trans-National Access (TNA) programme, organised with twelve nodes providing access to key experimental and data processing facilities, geographically distributed over Europe.

Guidelines are provided for creating a unified approach to the training of the TNA users across the individual PITHIA-NRF nodes, thus making the process more transparent and easier to navigate for those users. The training will be made using online resources, online events, and on-site training workshops.

 

The main objective of the strategy plan is to organize and implement effective dissemination, exploitation and communication activities. A basic element is the community building activity. It develops synergies with relevant international initiatives and advertise the project outcomes and the new capabilities that the use of integration tools offers to researchers, SMEs and R&D departments investing in technology. Systematic dissemination is planned towards the research community through the organization of high-level meetings, presentations at conferences, organization of scientific sessions in international conferences, and publications in highly ranked scientific journals. Outreach activities targeted to young students and to audience with scientific background but no specialization in space and atmospheric research and aim at informing them and attract their interest towards this specific research field. Demonstration experiments will be set up to explain the key physical processes such as the formation of auroral arcs and the HF communication principles. Finally, exploitation activities have as main objective the systematic collection of users’ feedbacks and the planning of potential upgrades in PITHIA-NRF tools and services. The best used PITHIA-NRF tools will be specified, and its marketing potential will be considered.

 

The Open Research Data of the H2020 PITHIA-NRF Project analyses the data management life cycle for the data to be collected, processed and/or generated by the project during its lifetime, and, where applicable, the provision of open access to research data. The Pilot will be revised and updated during the project when necessary. The contribution of all partners is critical for the proper and efficient data management.

 

PITHIA-NRF aims at establishing an interoperable network of infrastructures capable of proposing R&D services to expert and early-career researchers and to software and instrument development professionals. In order to achieve this, PITHIA-NRF develops the e-Science Centre as the central integration tool for providing data, models and scientific services, launches the Transnational Access programme, which is conducted in the framework of open calls for projects, and will organize three Innovation Days to discuss with expert stakeholders their needs and experience from using the PITHIA-NRF services.

PITHIA-NRF has the potential for enormous impact across several disparate fields of research and innovation, on the one hand, and on the areas of community, innovation, society, and politics, on the other. The successful implementation of the individual work-packages (WPs) will lead to the achievement of the expected impacts, whereas the influence of different WPs to the fulfilment of each impact, during the Project’s progress, is more direct or obvious than others. Therefore, PITHIA-NRF would greatly benefit from measuring and monitoring its progress and its subsequent impact during all Project’s works and tasks based on Key Performance Indicators (KPIs) adjusted to the WPs.

 

The PITHIA-NRF project aims to build a network of research facilities that specifically supports the research community related to the study of the Earth’s ionosphere, thermosphere and plasmasphere. An important key component of this infrastructure will be the PITHIA-NRF e-Science Centre, an integrated environment with a Web-based portal and underlying e-infrastructure resources to provide centralised access to all Datasets and Models/Applications associated with the PITHIA-NRF community, and to facilitate the integrated use of those for cross-national and cross-disciplinary use cases.

Design and implementation of such e-Science Centre is a complex process, both from the technical and also from the scientific perspective.  When designing such a solution, the detailed requirements of the targeted community have to be carefully explored and collected to assure that the final system satisfies their needs. On the other hand, the implementation also raises specific technical challenges as providing seamless access to a heterogeneous and distributed set of resources (databases and applications) is not a straightforward task. The aim of this document is to summarise the first steps that were taken towards designing and implementing the PITHIA-NRF e-Science Centre.

User requirements have already been collected in multiple steps and will be continued as an iterative process during the lifetime of the PITHIA-NRF project. Initial requirements were described in the project proposal and now part of the Description of Action (DoA). Formal and informal discussions have also been carried out between several members of the research community and the technical design team (primarily represented by the University of Westminster and EGI). Finally, a detailed questionnaire has been designed and distributed to all PITHIA-NRF partners.

In this document, we summarise the results so far and the current state of the requirements collection process. The major contribution is coming from the analysis of the questionnaires. At the time of writing this report, 59 answers were collected and analysed related to the various roles (Scientific User, Dataset Owner, Model/Application Owner) proposed in the e-Science Centre. Additionally, 16 individuals completed the generic section of the questionnaire that asks further questions related to the operation of the e-Science Centre. This set of answers is suitable and appropriate to start the design process and to make the most important decisions related to this process. However, the questionnaire remains “live” and we are expecting further stakeholders to complete it in the future. These additional answers can come from inside the project (e.g. in case a new Dataset or Model/Application becomes available), or even from outside the project from the wider user/provider community. These additional answers will be continuously monitored, analysed and the design will take these into consideration, wherever it is feasible and reasonable.

Based on the collected requirements, an initial design specification of the PITHIA-NRF e-Science Centre has been compiled. PITHIA-NRF Applications, Model and Datasets can either be deployed statically on PITHIA-NRF nodes or dynamically on e-infrastructure cloud compute resources, on demand. Therefore, the e-Science Centre architecture must support these two ways of accessing resources. The design specification identifies the major building blocks of the e-Science Centre, such as the Graphical User Interface (GUI), Knowledge Repository Exchange and Learning module (KREL), User Management system, Reference Architecture/Service Composer, Reference Architecture Launcher/Service Invocator, and the Reference Architecture/Service Repository. These components are described, and their desired functionalities are defined.

When implementing the e-Science Centre, we are planning to reuse existing components, technologies and frameworks as building blocks. Such modular design based on the customisation and further development of existing components is intended to fasten up the development process significantly. Therefore, we describe the identified candidate technologies and their state of the art, and explain how these technologies will be applied and extended when designing and implementing the PITHIA-NRF e-Science Centre. We also very specifically reflect on the design of the PITHIA-NRF data model, differentiating between scientific and technical metadata and suggesting and exploring standard formats for both.

Five reports are planned during the lifetime of the project that will describe the process and results of the implementation of the PITHIA-NRF e-Science Centre; this document is the first one. In order to provide a solid basis for the implementation process, this current document is essential with details of the requirements collection process, design specification and state of the art overview, as it was explained above.

 

This document aims to demonstrate the proof of concept implementation and the current status of the e-Science Centre in the PITHIA-NRF project. This proof of concept paves the way for the first prototype of the e-Science Centre.

The PITHIA-NRF project aims to build a network of research facilities that specifically supports the research community related to the study of the Earth’s ionosphere, thermosphere and plasmasphere (ITP). A key component of this infrastructure will be the PITHIA-NRF e-Science Centre, an integrated environment with a Web-based portal and underlying e-infrastructure to provide centralised access to the resources, more specifically Data Collections and Models, associated with the PITHIA-NRF community, and to facilitate the integrated use of these resources for cross-national and cross-disciplinary use cases.

Design and implementation of such e-Science Centre is a complex process, both from the technical and also from the scientific perspective. When designing such a solution, the detailed requirements of the targeted community have to be carefully elicited to ensure that the final system satisfies their needs. On the other hand, the implementation also raises specific technical challenges as providing seamless access to a set of heterogeneous and distributed Data Collections and Models is not a straightforward task. The aim of this document is to demonstrate a proof of concept implementation in order to illustrate how certain functionalities of the e-Science Centre would work, and present the work towards the e-Science Centre first prototype.

Specific e-Science Centre functionalities have been prioritised by the PITHIA community and the work presented in this document demonstrates proof of concept implementations based on those priorities. Those functionalities and priorities are a result of the user requirements collection process. The collection of user requirements has already started and it will continue as an iterative process until all the requirements have been collected and refined sufficiently. As part of this, a detailed questionnaire has been designed and distributed to all PITHIA-NRF partners and presented in Section 2 of the report titled “User requirements, state of the art and design specification”. Formal and informal discussions have also been taking place between the representatives of the research community and the technical design team (primarily represented by the University of Westminster and EGI) which has been informing the requirements as well as the implementation of the proof-of-concept. Initial requirements were described in the project proposal and they are now part of the Description of Action (DoA).

In this document, we present a proof of concept related to a Model that is deployed and executed on the Cloud, and report the work done towards the first prototype of the e-Science Centre. The first prototype will offer registration and search functionalities related to resources (Data Collections and Models) within the e-Science Centre. Three mechanisms have been identified for the registering and executing Models in the e-Science Centre. These are: i) Model executed on PITHIA site/node; ii) Model executed in the cloud; and iii) Model executed in user’s local environment. From these, the Cloud execution has been chosen for the proof of concept implementation due to the fact that it has been found that the community does not have much familiarity with this particular execution mechanism and its benefits. That proof of concept has been demonstrated to the community and feedback is included in this document.

Finally, we report the work completed towards the first prototype of the e-Science Centre, and the status of the current implementation. More specifically, we describe the technology selection process about the framework selected to build the e-Science Centre and the database that will store the related metadata, the design of the ontology and the metadata that will be used to register and search for resources (Data Collections and Models), current plans regarding the design and implementation of the registration and search processes, and the planned operational environment of the e-Science Centre. Regarding the functionalities of the first prototype of the e-Science Centre, the registration and search functionalities have been identified to be offered at this phase due the fact that those two functionalities are key services of the e-Science Centre and the community is eager to see and utilise them as soon as possible.

 

This document aims to analyse capabilities of and standards related to the PITHIA-NRF e-Science Centre. The objective is to provide the PITHIA-NRF community with a solid basis for the definitions and characteristics of the e-Science Centre resources (Data Collections and Models), present the resources collected so far, and describe the steps required so that Model owners can start working on the preparation of their Models in order to be able to register them before their subsequent registration with the e-Science Centre.

The PITHIA-NRF project aims to build a network of research facilities that specifically supports the research community related to the study of the Earth’s ionosphere, thermosphere and plasmasphere (ITP). A key component of this infrastructure will be the PITHIA-NRF e-Science Centre, an integrated environment with a Web-based portal and underlying e-infrastructure to provide centralised access to the resources, more specifically Data Collections and Models, associated with the PITHIA-NRF community, and to facilitate the integrated use of these resources for cross-national and cross-disciplinary use cases.

Design and implementation of such e-Science Centre is a complex process, both from the technical and also from the scientific perspective. When designing such a solution, the detailed requirements of the targeted community have to be carefully elicited to ensure that the final system satisfies their needs. On the other hand, the implementation also raises specific technical challenges as providing seamless access to a set of resources (Data Collections and Models), which are heterogeneous and distributed, is not a straightforward task. The aim of this document is to describe those resources, present the inventory of the resources collected so far, and provide guidelines for the preparation and deployment of Model in order to facilitate their registration with the e-Science Centre.

The collection of user requirements has already started and it will continue as an iterative process until all the requirements have been collected and refined sufficiently. Initial requirements were described in the project proposal and they are now part of the Description of Action (DoA). Formal and informal discussions have also been taking place between several members of the research community and the technical design team (primarily represented by the University of Westminster and EGI). Finally, a detailed questionnaire has been designed and distributed to all PITHIA-NRF partners and presented in Section 2 of the report titled “User Requirements, State of the Art and Design Specification”.

In this document, we report on the current state of the requirements relating to e-Science Centre resources (Data Collections and Models) and present the results of the work undertaken in relation to the definition and identification of the resources, namely Data Collections and Models. Furthermore, we provide guidelines for the preparation of Models so they can be registered and accessed through the e-Science Centre and executed with one of the available mechanisms: execution on PITHIA site/node, execution on cloud, and execution on users’ local resources. At the time of writing this document, 38 Data Collections and 12 Models have been identified as candidate resources that can be registered with the e-Science Centre. The Data Collections were analysed to understand the way they are currently accessed, to establish the different levels of processing applied to datasets and to identify where (i.e. observation facility) the data could come from. The Models were analysed to understand the way to access them and whether they can be executed on-demand. Additionally, the Models were associated with input Datasets (part of a Data Collections) wherever applicable. Those resources (Data Collections and Modes), even if not accessible at this very moment, are planned by the owners to be accessible through the e-Science Centre.

Based on the collected requirements, the initial design specification of the PITHIA-NRF e-Science Centre, and several formal and informal discussions with the community, it has been agreed that Models can be registered with the e-Science Centre and executed on-demand through one of the following execution mechanisms: execution on PITHIA site/node, execution on cloud, and execution on users’ local resources. This will require Model owners to prepare their Models for the respective mechanism(s). To assist Model owners prepare their Models for execution through those mechanisms, this document presents a set of guidelines for the Model owners to help them in the preparation of Models. A Model owner can choose to provide a Model in one or more of those mechanisms, and to assist such choice a comparison table has also been included.

 

PITHIA-NRF aims at establishing an interoperable network of infrastructures capable of proposing R&D services to expert and early-career researchers and to software and instrument development professionals. In the first year of the project the Consortium has worked to deliver the PITHIA-NRF vision and identity, the Project Management and Collaboration Tools, the Data Management Plan, the updated Communication, Outreach and Dissemination Strategy, and the ‘ethics requirements’, which all give the tools and set the rules for the project management, accessibility and extroversion. Moreover, regarding to:

• the integration activities, the e-Science Centre is designed and the proof of concept was released; the PITHIA-NRF capacities (data, metadata, research models and research facilities) were described; datasets and models to be registered to e-Science Centre, with relevant guidelines, were defined; the first WOOS is organized to set the foundations for the Optimization of Observing Strategies in the Network Facilities;
• the training activities, more than the expected workshops were organised (TPW#1, TPW#2, and two extra Technical Meetings); the PITHIA-NRF capacities, including data, metadata, research models and research facilities were defined;
• the dissemination activities, more than 10 papers were published and four presentations were given or accepted at conferences; the kick-off for the PITHIA-NRF e-newsletters distribution also took place;
• the communication activities, a comprehensive collection of information is released in the project web site and several events are planned to advertise the project and communicate the results to the stakeholders;
• the innovation activities, the First Innovation Meeting was postponed due to the Covid-19 pandemic; it is planned in June 2022 as a face-to-face event and several stakeholders are invited;
• the TNA activities, twelve projects are in progress and several new ones are expected to start, as the result of the 2nd TNA Call.

 

The near-Earth space environment undergoes daily changes driven by variable conditions in the Sun. Explosive eruptions of energy from the Sun causing minor solar storms on Earth are relatively common and of little consequence. On the contrary, rarely occurring superstorms generate physical changes in the Earth’s upper atmosphere detrimental to satellites, signals from global navigation systems, and radio systems.

While these events’ physics and engineering repercussions have been studied extensively, this is not the case for the related socioeconomic ramifications, despite the growing dependencies on these technologies. Therefore, the report identifies the infrastructures vulnerable to the upper atmosphere effects and quantifies their impacts on LEO satellites, systems offering PNT services, and radio systems through a systematic literature review.

[Report | Leaflet]

 

The PITHIA-NRF community Data Management policies are based on related policies and recommendations from the European Commission and EOSC. The major contribution is the effort to make PITHIA-NRF FAIR. All PITHIA-NRF facility nodes performed the FAIRness assessment, the gaps were analysed, and the way that design and development of the e-Science Centre can help PITHIA-NRF achieve the FAIR requirements was discussed. all PITHIA-NRF participation RIs discussed the action plan to implement the policies as well.

 

Advance model developments require quality-controlled data to provide reliable results. A key topic to discuss in PITHIA-NRF is the standards for data quality control and data management to be used by the participating RI facilities. We aim to specify the higher-level data products based on the policies and recommendations provided by the Research Data Alliance, ESFRI, EOSC and the domain specific methodologies, and related reports and white papers are reviewed. We also surveyed the current status of PITHIA-NRF facility RIs on quality control and tools used for curations and quality processes.

We attempt to establish a common reference framework, identifying major areas concerning data quality that can be used by PITHIA-NRF data providers to assess the data quality when generating and publishing data. The major contribution includes the definition of the Data Quality Flag that indicates the scientific quality of research data which is a challenging issue not only for PITHIA-NFR but also for many other European Research Infrastructures. We involved PITHIA-NRF data providers in the discussion in order to achieve consensus. In the future work, we will endeavour on how to implement the quality assessment in the design and development of PITHIA-NRF ontology and e-Science Centre, and how to apply in daily practices of individual PITHIA-NRF data providers

 

The main concept of PITHIA-NRF is to integrate key research facilities and provide efficient access for researchers to facilitate research advances and transition of data products and models from research to application and to innovation. The PITHIA-NRF e-Science Centre is being developed for the efficient curation, preservation and provision of access to the data collected or produced under the project. The PITHIA-NRF Knowledge Book offers high-level information on the infrastructures, data and models provided through the e-Science Centre, so that the users can appreciate the strength and depth of the PITHIA-NRF network and the services that are provided.

 

Selected courses from the Training for Partners workshops, demonstrating the use of key integration tools were recorded and made available through the public website in the form of webinars.

 

The design and development of the PITHIA e-Science Centre (eSC) started in April 2021 and progressed through 2022 and early 2023 to result in the first official prototype of the system. The eSC is openly available at https://esc.pithia.eu/. As the eSC is under continuous development, the document provides a snapshot of the portal as of March 2023. The screenshots are generated from the particular version that was available at that time. Therefore, visiting the official eSC website at a later point of time will result in seeing a newer, more advanced version. The document shortly outlines the motivation and design principles of the system, the work on the ontology and metadata structure that underlines its structure, a detailed description of the currently available functionalities, ongoing work regarding its further development, also its production deployment and operation. At the end of the report the next steps are described.

 

The report provides a snapshot of the deployed data sets in the e-Science Centre at the date of release. It is a living document being updated over the course of the project.

 

The report describes the Trans-National Access (TNA) activities during the first two years of the PITHA-NRF project. It contains the procedures used for selecting and monitoring the different TNA projects in order to ensure their scientific excellence. It also contains an overview of the TNA projects from the first three TNA calls, and of the activities of the PITHIA Access Bureau (PAB) that was established at an early point in the PITHIA-NRF project to oversee the work on the TNA related activities.

 

PITHIA-NRF aims at establishing an interoperable network of infrastructures capable of proposing R&D services to expert and early-career researchers and to software and instrument development professionals.

In the first two years of the project (months 1-24) the Consortium has worked to deliver the PITHIA-NRF vision and identity, the Project Management and Collaboration Tools, the Data Management Plan, the first release of the e-Science Centre, the updated Communication, Outreach and Dissemination Strategy, and the “ethics requirements”, which all give the tools and set the rules for the project management, accessibility and extroversion. Moreover, regarding to:

• the integration activities, the e-Science Centre is designed, the proof of concept was released and the first release of the e-Science Centre with open access was published; the PITHIA-NRF capacities (data, metadata, research models and research facilities) were described; datasets and models to be registered to e-Science Centre, with relevant guidelines, were defined and deployed; three WOOS were organized for the Optimization of Observing Strategies in the Network Facilities;
• the training activities, more than the expected workshops were organised (TPW#1, TPW#2, TPW#3, TPW#4 and two extra Technical Meetings); the PITHIA-NRF capacities, including data, metadata, research models and research facilities were defined and included in the Knowledge Book;
• the dissemination activities, 29 papers were published and 39 presentations were given or accepted at a plethora of conferences; the newsletters are distributed every 6 months; the first High Profile Meeting was organised as a hybrid event and several stakeholders were participated;
• the communication activities, a comprehensive collection of information is released in the project web site and several events are planned to advertise the project and communicate the results to the stakeholders;
• the innovation activities, the First & Second Innovation Meeting were organised as a face-to-face and online event and several stakeholders were participated;
• the TNA activities, four Calls were released, in the frame of which 29 applications were submitted and projects were implemented or are in progress; several new TNA projects are expected to start, as the result of the 4.

 

Selected courses from the First and Second Training Schools have been selected with the scope of providing knowledge tools to the research users and the reference community and have been made available through the public website in the form of webinars.

 

The e-Science Centre of the PITHIA-NRF project has been released, and is available for owners of data sets and models related to the thermosphere, ionosphere, or plasmasphere to register there data collection in order to make them more accessible to users. A user manual is available describing the  technical  details  of  this registration  process.  The  current document  presents  in  addition guidelines and best practices to help owners of data collections decide what resources they should register.  A few scenarios are described as examples to help data owners make those decisions.

These guidelines are derived from experiences of consortium partners during the registration of their data, and from discussions during the training for partner workshops organised in the framework of the project.

 

The report presents the outcomes of the tasks aimed at maintaining and advancing standardisation processes for software and high-level data products critical to PITHIA-NRF observations. The report reviews the standard-making procedures vital to the PITHIA-NRF community. It highlights ongoing efforts related to data product accuracy, validation, and compliance with European standards and solutions implemented to standardise the data registration process as well as the registration and execution of models implemented in the e-Science Centre. 

The data quality level characterisation in the PITHIA-NRF project plays a vital role in ensuring that research and modelling efforts are based on accurate and reliable data. The standardised way of registering and characterising data and models in the PITHIA-NRF e-Science Centre enables researchers to make informed choices about the datasets they use, enhances the accuracy of models, and supports the integrity and reproducibility of scientific findings. By implementing data quality assessments, and in developing higher-level data products in collaboration with users the PITHIA-NRF project ensures that its outputs are robust, credible, and valuable for the scientific community.