On the Limits of Contemporary Academic Evaluation Models: The Obsession with Indexes and Metrics (by F. R. Cunha – ENM/FT/UnB)

Contemporary academic evaluation models often focus heavily on indexes and metrics to assess research quality and impact. This reliance has created a system where numerical values and rankings overshadow other meaningful aspects of scholarly work. While indexes and metrics provide a convenient way to compare and categorize academic output, they often fail to capture the full scope and significance of research contributions. Important qualitative elements, such as originality, societal relevance, and interdisciplinary impact, are frequently overlooked in favor of easily measurable indicators.

There is little doubt that the CAPES evaluation system has imposed an extremely demanding and, in several respects, distorted logic upon the academic activity of the contemporary professor-researcher. Today, faculty members are expected to publish continuously in high-impact journals, teach undergraduate and graduate courses, participate in administrative activities, supervise undergraduate and graduate students, develop technological innovation, review scientific papers, serve as editors, attend international conferences, and simultaneously accumulate awards and academic distinctions. The outcome of this multiplicity of demands is an evident scenario of overload. Faculty members strongly engaged in research are frequently subjected to exhausting work routines, far beyond those experienced by colleagues who do not follow the same academic trajectory, despite operating under broadly equivalent institutional conditions. The increasing intensification of evaluative criteria imposed by CAPES, often implemented in a unilateral manner, has generated substantial pressure on faculty members, potentially compromising the sustainability and critical mass of graduate programs.

This context raises legitimate concerns, since the continuous intensification of quantitative requirements may discourage the retention and engagement of highly productive researchers, negatively affecting academic cohesion, intellectual diversity, and the training capacity of graduate programs. There is also a broader structural issue that deserves attention: the growing obsession with metrics, percentiles, FWCI, H-indexes, and quartile classifications (Q1, Q2, etc.), a phenomenon particularly pronounced in developing countries. A culture of dependency on these indicators is gradually established, as though they were ends in themselves, leading many researchers to behave as mere “metric performers” rather than scientists driven by relevant scientific and societal problems.

| have consistently maintained my position on this matter. | continue to conduct my research and supervise students in the same way | always have: with consistency, dedication, and concrete results. | have held a CNPq productivity fellowship since 1996 and have been evaluated at the highest levels in recent years, yet | have never regarded CAPES as the central priority guiding my scientific agenda. Blindly following such directives is, in many ways, analogous to rigidly adhering to an instruction manual without exercising critical judgment; an attitude that is ultimately counterproductive in science.

Another recurring problem concerns the instability of the evaluation criteria themselves. In previous years, the JCR impact factor played a central role and it still appears prominently in the Lattes curriculum system, but it has gradually lost prominence within CAPES evaluations. What emerges instead is a continuous reshaping of the rules, a kind of “friendly fire,” in which the level of demand is progressively raised, almost as if the implicit goal were to reduce the number of active programs and redistribute increasingly scarce resources among a smaller number of survivors. Meanwhile, the number of graduate programs continues to expand without a proportional increase in funding.

In discussions with international colleagues, this evaluation logic has never been fully understood. In many academic environments abroad, teaching quality is also valued in a substantial and meaningful way, something frequently neglected within this evaluative framework. After all, we are university professors, not merely researchers working in institutes exclusively dedicated to research. In international practice, metrics such as the H-index and impact factor are still considered, but they rarely determine publication strategies in an absolute sense. Researchers at institutions such as Cambridge continue publishing in the leading journals of fluid mechanics regardless of percentile classifications. Classical journals such as Journal of Fluid Mechanics, Physics of Fluids, Journal of Rheology, Journal of Non-Newtonian Fluid Mechanics, and Soft Matter remain central to the scientific community, even when classified as P3 or P4 according to certain CAPES criteria. The same occurs in specialized areas such as ferrofluids, where the Journal of Magnetism and Magnetic Materials continues to be regarded as a leading reference journal. The idea of “journal hunting” based exclusively on metrics is inconsistent with a solid and mature scientific posture. Researchers should publish primarily where their work has thematic relevance, scientific significance, and genuine impact within the specialized community. | will continue submitting papers to the established journals in my field because these are the venues where there is the greatest likelihood of qualified readership, academic recognition, and meaningful citation impact.

Furthermore, we cannot allow a system such as CAPES to undermine scientific interaction within graduate programs themselves. Such an outcome would represent a significant setback, contrary to the very essence of scientific development, which fundamentally depends on collaboration, exchange of ideas, and collective intellectual construction among faculty members and students. Finally, in the program self-evaluation document, | also registered an important recommendation: avoiding premature analyses during the first year of the four-year evaluation cycle. | suggested that assessments should be conducted cumulatively starting from the second year (end of 2026), when a more robust and representative dataset will be available to guide strategic adjustments.

Overall, this is a system marked by several inconsistencies and based on metrics that exhibit well-known distortions and limitations. This reinforces the importance of critical thinking, intellectual autonomy, and a firm positioning on the part of researchers. Adopting the percentile-based classification system (P1—-P8), according to the established intervals, it becomes possible to analyze more systematically the set of journals in which | have published or intend to publish. This analysis reveals a scientific production concentrated predominantly in the upper strata (P1—P3), including internationally recognized journals in fluid mechanics and rheology such as Journal of Fluid Mechanics, Physics of Fluids, and Journal of Non-Newtonian Fluid Mechanics. This panorama reinforces an important point: the scientific quality and impact of a research work cannot be reduced exclusively to percentile metrics. Many of the most influential and historically established journals in the field of fluid mechanics lie outside the P1 category, highlighting the inherent limitations of this type of classification. Moreover, my publication strategy has been guided primarily by thematic adherence and the potential for genuine scientific impact within the community, rather than merely by quantitative metrics. This approach is consistent with consolidated international academic practices and contributes more effectively to the visibility, relevance, and long-term significance of the research being developed.

The domination of indexes and metrics within academic evaluation models highlights their limitations and calls for a more balanced approach. Recognizing the drawbacks of this obsession is essential for creating a system that values diverse contributions and fosters genuine academic progress.