A Usage Observatory to learn more about the practices of OpenEdition users
The Usage Observatory research project is part of the umbrella Equipex+ COMMONS project. Its goal is to gather and analyse data on how users navigate on OpenEdition’s platforms, to gain insight into usage and make resources more accessible to users. Our colleagues at OpenEdition Lab – Simon Dumas Primbault (S.D.P.), CNRS junior professor of open science in HSS and manager of the Lab, Ioanna Faïta (I.F.), a doctoral student in information and communication sciences, and Mohsine Aabid (M.A.), a doctoral student in computer science – remind us of the specifics of the project and provide an overview of the first set of data collected.
- The Usage Observatory is part of the Lab’s first key research area, the goal of which is to gather data on open science platform users and their information practices. How is this project being rolled out? And how will it benefit OpenEdition’s user community and, more broadly, those of Huma-Num and Métopes, the other two infrastructures involved in the COMMONS project?
S.D.P. The COMMONS project was drawn up in 2020 [before S.D.P. joined OpenEdition] in response to the Equipex+ call for expressions of interest. At the time, the Usage Observatory was conceived as a tool for guiding the technical development of the consortium, the aim being to identify the “needs” of the consortium’s “target audiences” and deliver a “seamless user experience”, especially when navigating from one infrastructure to another. The initial desired outcome was therefore very operational in nature, falling within the realm of software engineering (UX/UI). Then, in the wake of the Open Knowledge Appropriation and Umberto projects, it was decided that two doctoral students should join OpenEdition Lab (a research hub) as the observatory team. This gave rise to another desired outcome based on deeper reflection and critical analysis1 – the raison d’être of research work – which included revealing usages in all their anthropological depth, understanding the dynamics of an open science ecosystem, questioning the notion of data, and so on.
The challenge for Ioanna and Mohsine with this approach is to find the right balance between the observatory’s operational and reflective aspects. Regarding the former, for the research work to produce operational outcomes, the team will need to rely on external partners (including the other services within OpenEdition, or service providers) to help them turn their PhD findings into concrete solutions, which could be anything from simply changing the position of a button in an interface, to prototyping dedicated navigation tools.
As for the reflective aspect, Ioanna and Mohsine’s work will lead to a whole range of future opportunities. From a methodological perspective, combining the application of computational methods to repurpose the use of digital footprints with a qualitative approach based on observations and interviews will pave the way for developing new interdisciplinary tools, to hopefully gain insights into the complex matter of informational usages, especially in a digital context (see below).
From a sociotechnical perspective, carrying out research on and within an infrastructure like OpenEdition, and, more broadly speaking, a consortium like COMMONS, places you at the intersection between standards and practices, policies and usages, and institutions and communities. This is precisely where discussions are taking place about what open science is, and how it is shaping scientific practices as well as the relationships between science and society. From that point of view, the Usage Observatory will provide us with precious insights into what an open science infrastructure should be – its governance and its role within a larger ecosystem on which it depends and vice versa.
- What does taking a computational approach to user paths mean?
M.A. Taking a computational approach to user paths means using computational methods and tools to model and analyse the paths taken by platform users as they interact with online products or services. The idea is to collect the data generated when a so-called client (user) computer submits a query (requests a resource) to a server. The data is saved in a file to be analysed at a later date. The goal of my research is to document the practices of users visiting OpenEdition’s platforms. One of the tasks is to retrace their steps, try to understand their choices, and figure out why they decided to follow one path over another. It will allow us to provide users with navigation tools that generate more relevant results during their browsing sessions.
- What did the first analysis reveal?
M.A. There are no conclusive results for the moment. Data mapping offers an overview of the distribution of the sites according to users. However, the results are unstable, so we are working to improve on that aspect. Our analysis of user paths has nevertheless revealed four main navigation profiles so far, which all need to be studied in much more detail. In terms of methodology, our analysis has allowed us to learn more about data processing techniques, the user session segmentation process, and the cleaning of logs. To obtain concrete results, we will be testing our approach on a larger data set.
- What does the preliminary exploratory study consist of and what are its goals? Can we already draw some initial conclusions?
I.F. We launched a quantitative and qualitative preliminary exploratory study in March. It allowed me to draw on the reading from my first year of thesis to work on developing and validating a methodology that others could then follow, and, of course, adjust if necessary. We have just finished the quantitative part of the study, which is built around a survey protocol applied to three cases: two journals and one publisher (or three book series, to be precise) whose access modes have been changed. The scope is limited for now as the goal is to scale up over the next few years by observing and documenting all the websites hosted on OpenEdition.
The survey protocol consists of a questionnaire created using the free online software application Framaform. After meeting up with each publishing team individually, I managed to design a questionnaire that took their specific needs into account while remaining universal enough to be used in all three cases. The form/questionnaire was used to help collect preliminary, therefore quantitative, information on reading practices and readers’ perceptions of the journal/publisher. But it also doubles up as a means of recruiting survey respondents for the qualitative part of the study, which will close the loop for the survey protocol.
According to the original schedule, the online survey was supposed to last until the end of July, which was also supposed to mark the end of the first cycle. However, not enough people had responded to the survey at the time, so I had to extend the deadline. Dealing with that kind of issue is also part of testing the methodology: it made us think about why so few people answered the survey. We concluded that the problem was the way in which the survey was disseminated and the fact it wasn’t visible enough on the platforms concerned – this is one of the field-related issues we will need to set straight to make sure the research project is a success.
- Which survey techniques served as a basis for the qualitative approach?
I.F. Our qualitative approach is based on survey techniques taken from sociology, ethnography and semiotics. The observational part of the study, which involves examining OpenEdition platform interfaces, such as document indexing, search tools and content indexing, has been underway since my thesis began. It is a process that has drawn heavily on my previous experience within the infrastructure, as an editorial officer for OpenEdition Books and Journals. As for the digital ethnography part, which consists in applying ethnography methods to improve our understanding of user behaviour in digital environments, I will be initiating a round of semi-structured interviews next month as the next logical step after the quantitative part of the exploratory study.
The round of interviews will include three different formats so that all the stakeholders can be interviewed: users, which goes without saying, but also publishing teams and the intermediary in charge of disseminating Freemium content. The idea is to interview the people previously surveyed about their practices and their perceptions of the journal or publisher, as well as the intermediary – the OpenEdition platform. This part of the study is unlikely to be completed until early 2025, given the amount of time needed to transcribe the interviews.
- What are the next key dates for the Observatory and the Lab?
S.D.P. The exploratory study carried out in 2024, within well-defined scopes that are familiar to Ioanna and Mohsine, will naturally provide the two journals and the publishing house with some answers to their questions about their readership. However, this study is above all an opportunity to test survey and analysis tools and techniques, thereby evaluating their effectiveness and relevance. The next step would be to devise a more universal survey protocol, which we will first apply within OpenEdition. Therefore, once the technical and methodological report has been completed for this first field study in early 2025, Ioanna and Mohsine will need to think about designing a large-scale protocol that is coherent, adjustable and replicable, with the idea of meeting operational expectations as well as continuing to explore issues related to their own research questions.
As for OpenEdition Lab more generally, we had the pleasure of welcoming a new doctoral student in early October. Marcella Araujo Malheiros will work together with Professor Lise Verlaet from the University of Montpellier 3 on the discoverability of open access HSS content online, especially with the sudden disruptive rise of generative AI.
The PaRéDo SHS project, for which OE Lab and its partner, the Centre for Internet and Society, were awarded funding by the French National Research Agency, will be launched at the start of November. A one-day workshop on the governance of digital knowledge infrastructures will be held in connection with this at the Condorcet Campus. Otherwise, a postdoctoral researcher and a research engineer will be joining the PaRéDo SHS team to help carry out research on data sharing and reuse practices in HSS. They will also be looking at how such practices, regardless of shape or form, can be fostered by specific types of governance structure – will there be a move towards more data commons?
Interested in finding out more?
Read these posts in French from the Lab’s blog:
- Mohsine’s post: Voyages Virtuels : Première analyse computationnelle des habitudes de navigation sur OpenEdition (Virtual Journeys: Initial Computational Analysis of User Navigation Habits on OpenEdition)
- Ioanna’s post: Observatoire des usages COMMONS : La nécessité d’une première étude exploratoire (The COMMONS Usage Observatory: the Need for a Preliminary Exploratory Study)
And this post in French about the PaRéDo SHS project: https://lab.hypotheses.org/paredo-shs-2024-2027
- As well as an educational outcome, given that a PhD is a university course leading to a degree. ↩︎
OpenEdition suggests that you cite this post as follows:
OpenEdition team (October 29, 2024). A Usage Observatory to learn more about the practices of OpenEdition users. Open Electronic Publishing. Retrieved November 9, 2024 from https://doi.org/10.58079/12l6z