The issue of plagiarism—claiming credit for work that is not one’s own, rightly, continues to cause concern in the academic community. An analysis is presented that shows the effects that may arise from metrics-based assessments of research, when credit for an author’s outputs (chiefly publications) is given to an institution that did not support the research but which subsequently employs the author. The incentives for what is termed here “institutional plagiarism” are demonstrated with reference to the UK Research (...) Assessment Exercise in which submitting units of assessment are shown in some instances to derive around twice the credit for papers produced elsewhere by new recruits, compared to papers produced ‘in-house’. (shrink)

Because ports are considered to be the heart of the maritime transportation system, thereby assessing port performance is necessary for a nation’s development and economic success. This study proposes a novel metric, namely, “_port performance index (PPI)_”, to determine the overall performance and utilization of inland waterway ports based on six criteria,_ port facility, port availability, port economics, port service, port connectivity, and port environment_. Unlike existing literature, which mainly ranks ports based on quantitative factors, this study utilizes (...) a Bayesian Network (BN) model that focuses on both quantitative and qualitative factors to rank a port. The assessment of inland waterway port performance is further analyzed based on different advanced techniques such as sensitivity analysis and belief propagation. Insights drawn from the study show that all the six criteria are necessary to predict PPI. The study also showed that port service has the highest impact while port economics has the lowest impact among the six criteria on PPI for inland waterway ports. (shrink)

Novelty is a key concept to understand creativity. Evaluating a piece of artwork or other creation in terms of novelty requires comparisons to other works and considerations about the elements that have been reused in the creative process. Human beings perform this analysis intuitively, but in order to simulate it using computers, the objects to be compared and the similarity metrics to be used should be formalized and explicitly implemented. In this paper we present a study on relevant elements (...) for the assessment of novelty in computer-generated narratives. We focus on the domain of folk-tales, working with simple plots and basic narrative elements: events, characters, props and scenarios. Based on the empirical results of this study we propose a set of computational metrics for the automatic assessment of novelty. Although oriented to the implementation of our own story generation system, the measurement methodology we propose can be easily generalized to other creative systems. (shrink)

The core issue of our target article concerns how relational complexity should be assessed. We propose that assessments must be based on actual cognitive processes used in performing each step of a task. Complexity comparisons are important for the orderly interpretation of research findings. The links between relational complexity theory and several other formulations, as well as its implications for neural functioning, connectionist models, the roles of knowledge, and individual and developmental differences, are considered.

ObjectiveThere is little literature on the validity of kurtosis-adjusted noise energy metrics in human studies. Therefore, this study aimed to validate the application of cumulative noise exposure adjusted by kurtosis in evaluating occupational hearing loss associated with non-Gaussian noise among manufacturing workers.MethodsA cross-sectional survey was conducted on 1,558 manufacturing workers exposed to noise from five industries to collect noise exposure and hearing loss data. Both CNE and kurtosis-adjusted CNE were collapsed into 2-dB∙year bins, and the mean noise-induced permanent threshold (...) shifts at 3, 4, and 6 kHz in each bin were calculated. The contributions of CNE and CNE′ to noise-induced hearing loss were compared using the multiple linear regression. The degree of overlap of two linear regression equations was used to evaluate the validity of the CNE′ using a stratified analysis based on age and sex.ResultsMultiple linear regression models showed that after kurtosis adjustment, the standardized regression coefficient of CNE increased from 0.230 to 0.255, and R2 increased from 0.147 to 0.153. The linear relationship between NIPTS346 and CNE′ or CNE showed that the regression line of non-Gaussian noise was closer to that of Gaussian noise when using CNE′ than using CNE. The mean difference in NIPTS346 between the equations of non-Gaussian noise and Gaussian noise was significantly reduced from 4.32 to 1.63 dB HL after kurtosis adjustment. Through a stratified analysis, these significant decreases were observed in male and female workers, and workers aged ≥30 years old.ConclusionAs a noise exposure metric combining noise energy and temporal characteristics, the kurtosis-adjusted-CNE metric was more effective than CNE alone in assessing occupational hearing loss among manufacturing workers in non-Gaussian noise environment. However, more studies are needed to verify the validity of the kurtosis-adjusted-CNE metric. (shrink)

Automated neuropsychiatric batteries have been used in research and clinical practice, including for chronic diseases, such as Systemic Lupus Erythematosus. The Pediatric Automated Neuropsychological Assessment Metrics battery, originally developed for use in American-English speaking individuals, allows tracking of cognitive functions. It can be applied to people over 9 years old. The aim of this study was to translate and present initial validation data from the Ped-ANAM into Brazilian-Portuguese. We translated the battery according to Beaton’s guidelines. Psychometric properties were (...) tested, internal consistency was analyzed by Cronbach’s alpha coefficient, test-retest reliability by the intraclass correlation coefficient. Further, we measured the test execution speed at both times as a temporal stability. Principal component analysis was used for structural validity. Evidence of construct validity was assessed through assessment of the relationships with the Wechsler Intelligence Scales. All participants prior to the start of study related activities signed an informed consent form approved by the local ethics committee. A sample of 230 individuals [mean of age: 23 years; 65% females] was included; a subset of 51 individuals [mean of age: 18 years, 59% female] completed the Ped-ANAM twice to assess test-retest reliability, and another subset of 54 individuals [mean of age: 20.4 years; 67% female] completed the Wechsler Intelligence Scales for Children and Adult for assessment of the Ped-ANAM’s construct validity. Our results suggest that the internal consistency of the Ped-ANAM and its subtest test-retest reliability were excellent. There was no clustering in the Principal Components Analysis, suggestive of non-grouping of the evaluated variables. Construct validity assessment to the Wechsler Scales showed expected ranges of low to strong correlations. We concluded that, based on the results of this study, a cross-culturally validated Brazilian-Portuguese version of the Ped-ANAM has been developed and it is a reliable tool for the screening cognitive function. (shrink)

Cloud computing (ClC) has become a more popular computer paradigm in the preceding few years. Quality of Service (QoS) is becoming a crucial issue in service alteration because of the rapid growth in the number of cloud services. When evaluating cloud service functioning using several performance measures, the issue becomes more complex and non-trivial. It is therefore quite difficult and crucial for consumers to choose the best cloud service. The user's choices are provided in a quantifiable manner in the current (...) methods for choosing cloud services. Hence, this study attempts to achieve this objective through construction. decision-making framework so-called cloud services evaluator (CSsEv). The main indicator and its sub-indicators are formed as nodes at levels(n) in tree soft sets (TSSs). Thereafter Single Value Neutrosophic Sets (SVNSs) as branch of neutrosophic sets which conjunction with the Multi-Criteria Decision Making (MCDM) technique to facilitate analysis and evaluation process for the available Cloud services providers. Hence, entropy is employed to obtain indicators and sub_indicators weights and Complex Proportional Assessment utilizes these weights to facilitate the decision process of selecting optimal ClSPs. (shrink)

Neurodegenerative Parkinson's Disease (PD) is one of the common incurable diseases among the elderly. Clinical assessments are characterized as standardized means for PD diagnosis. However, relying on medical evaluation of a patient's status can be subjective to physicians' experience, making the assessment process susceptible to human errors. The use of ICT-based tools for capturing the status of patients with PD can provide more objective and quantitative metrics. In this vein, the Personalized Serious Game Suite (PGS) and intelligent (...) Motor Assessment Tests (iMAT), produced within the i-PROGNOSIS European project (www.i-prognosis.eu), are explored in the current study. More specifically, data from 27 patients with PD at Stage 1 (9) and Stage 3 (18) produced from their interaction with PGS/iMAT are analyzed. Five feature vector (FV) scenarios are set, including features from PGS or iMAT scores or their combination, after also taking into consideration the age of patients with PD. These FVs are fed into three machine learning classifiers, i.e., K-Nearest Neighbor (KNN), Support Vector Machines (SVM), and Random Forest (RF), to infer the stage of each patient with PD. A Leave-One-Out Cross-Validation (LOOCV) method is adopted for testing the classification performance. The experimental results show that a high (>90%) classification accuracy is achieved from both data sources (PGS/iMAT), justifying the effectiveness of PGS/iMAT to efficiently reflect the motor skill status of patients with PD and further potentiating PGS/iMAT enhancement with a machine learning a part to infer for the stage of patients with PD. Clearly, this integrated approach provides new opportunities for remote monitoring of the stage of patients with PD, contributing to a more efficient organization and set up of personalized interventions. (shrink)

This article presents a novel full-reference image quality assessment algorithm by depicting the sub-band characteristics in the wavelet domain. The proposed image quality assessment method is based on energy estimation in the wavelet-transformed image. Image QA is achieved by applying a multilevel wavelet decomposition on both the original and the enhanced image. Next, the wavelet energy and vector are computed to obtain the percentage of the energy that corresponds to the approximation and the details, respectively. Further, the (...) approximate and detailed energy levels of both the original and the enhanced images are compared to formulate an image quality assessment. Numerous experiments are conducted on a dozen of image enhancement algorithms. The results presented show that the image with poor contrast in the foreground than the background has continuous regular coefficient values. The probability density function for such an image has a relatively lower WE and skewness compared with the background. The proposed scheme not only evaluates the global information of an image but also estimates the fine, detailed changes in an enhanced image. Thus, the proposed metric serves as an objective and effective FR criterion for color image QA. The experimental results presented confirm that the proposed WE metric is an efficient and useful metric for evaluating the quality of the color image enhancement. (shrink)

Kovacs calls for collaborating teams to indicate the proportional credit that each author of a multi-authored paper deserves.1 This approach addresses the problem of giving each of the co-authors full credit for the article when their publication records are assessed. This problem is, however, a weakness in the evaluation system, not in the publication system, and it will not be solved by the proposed strategy.As the author notes, publication records are critical to decisions on hiring, promotion, tenure, salaries and allocation (...) of research resources. At each of these points, what matters most is the quality of the candidate's work, which cannot be adequately assessed by quick counts of articles, even when the counts are weighted by numbers of citations or impact factors. Some institutions and departments perform careful reviews; others …. (shrink)

De Marchi and Lorenzetti :253-261, 2016) have recently argued that in fields where the journal impact factor is not calculated, such as in the humanities, it is key to find other indicators that would allow the relevant community to assess the quality of scholarly journals and the research outputs that are published in them. The authors' suggestion is that information concerning the journal's rejection rate and the number of subscriptions sold is important and should be used for such assessment. (...) The question addressed by the authors is very important, yet their proposed solutions are problematic. Here I point to some of these problems and illustrate them by considering as a case in point the field of philosophy. Specifically, here I argue for four main claims. First, even assuming that IF provides a reliable indicator of the quality of journals for the assessment of research outputs, De Marchi and Lorenzetti have failed to validate their suggested indicators and proxies. Second, it has not been clarified why, in absence of IF, other journal-based metrics that are currently available should not be used. Third, the relationship between IF and rejection rate is more complex than the authors suggest. Fourth, accepting the number of sold subscriptions as a proxy would result in discrimination against open access journals. The upshot of my analysis is that the question of how to assess journals and research outputs in the humanities is still far from resolved. (shrink)

Subjective, multi-attribute choice decisions – such as whom to marry or which college to attend – play a substantial role in decision makers’ long-term well-being. However, the metacognition literature lacks tools for assessing metacognitive capacities in subjective decisions. We present three studies in which we propose and validate the Knowledge of Weights (KoW) paradigm, a novel method for assessing metacognitive knowledge of attribute weights in subjective, multi-attribute choice decisions. In Study 1, we demonstrate the test-retest reliability of metrics generated (...) by the KoW paradigm. In Study 2, we apply the KoW paradigm in four domains and show that it generates consistent results. In Study 3, we demonstrate that participants who perform better on the KoW paradigm make choices with which they are more satisfied, providing suggestive evidence of predictive validity. Use cases in cognitive psychology and beyond are discussed. (shrink)

With growth in the quantity of business ethics journals in recent years, assessments of journal quality are helpful to ethics researchers and administrators, as researchers consider available publication venues, and administrators consider the value of faculty research. The few published evaluations of business ethics journals have predominantly utilized two methods of journal quality determination: citation analysis and surveys of active researchers. This study employs a novel method to assess business ethics journals: 83 Association to Advance Collegiate Schools of Business business (...) schools provided their internally developed journal lists that were used to evaluate faculty research, and the submitted lists were then analyzed for the presence and assessment of business ethics journals. This analysis yielded a ranking of 24 business-ethics-centric journals, and this ranking reflects the collective judgments of AACSB business school faculties. The results of this study are pragmatic in that the journal evaluation data employed metrics actually used by business schools to determine the quality of business ethics journals. These findings also provide additional impetus for the recognition of business ethics as a distinct business discipline and business ethics research as a unique field of scholarly endeavor. While studies of business ethics may be influential when they are published in non-BEC journals, such studies may be more powerfully impactful when published in BEC journals. (shrink)

User modeling is a key topic in many applications, mainly social networks and information retrieval systems. To assess the effectiveness of a user modeling approach, its capability to classify personal characteristics (e.g., the gender, age, or consumption grade of the users) is evaluated. Due to the fact that some of the attributes to predict are multiclass (e.g., age usually encompasses multiple ranges), assessing fairness in user modeling becomes a challenge since most of the related metrics work with binary attributes. (...) As a workaround, the original multiclass attributes are usually binarized to meet standard fairness metrics definitions where both the target class and sensitive attribute (such as gender or age) are binary. However, this alters the original conditions, and fairness is evaluated on classes that differ from those used in the classification. In this article, we extend the definitions of four existing fairness metrics (related to disparate impact and disparate mistreatment) from binary to multiclass scenarios, considering different settings where either the target class or the sensitive attribute includes more than two groups. Our work endeavors to bridge the gap between formal definitions and real use cases in bias detection. The results of the experiments, conducted on four real-world datasets by leveraging two state-of-the-art graph neural network-based models for user modeling, show that the proposed generalization of fairness metrics can lead to a more effective and fine-grained comprehension of disadvantaged sensitive groups and, in some cases, to a better analysis of machine learning models originally deemed to be fair. The source code and the preprocessed datasets are available at the following link: https://github.com/erasmopurif/toward-responsible-fairness-analysis. (shrink)

In climate science, observational gridded climate datasets that are based on in situ measurements serve as evidence for scientific claims and they are used to both calibrate and evaluate models. However, datasets only represent selected aspects of the real world, so when they are used for a specific purpose they can be a source of uncertainty. Here, we present a framework for understanding this uncertainty of observational datasets which distinguishes three general sources of uncertainty: (1) uncertainty that arises during (...) the generation of the dataset; (2) uncertainty due to biased samples; and (3) uncertainty that arises due to the choice of abstract properties, such as resolution and metric. Based on this framework, we identify four different types of dataset ensembles — parametric, structural, resampling, and property ensembles—as tools to understand and assess uncertainties arising from the use of datasets for a specific purpose. We advocate for a more systematic generation of dataset ensembles by using these sorts of tools. Finally, we discuss the use of dataset ensembles in climate model evaluation. We argue that a more systematic understanding and assessment of dataset uncertainty is needed to allow for a more reliable uncertainty assessment in the context of model evaluation. The more systematic use of such a framework would be beneficial for both scientific reasoning and scientific policy advice based on climate datasets. (shrink)

Exception from Informed Consent (EFIC) regulations detail specific circumstances in which Institutional Review Boards (IRB) can approve studies where obtaining informed consent is not possible prior to subject enrollment.To better understand how IRB members evaluate community consultation (CC) and public disclosure (PD) processes and results, semi-structured interviews of EFIC-experienced IRB members were conducted and analyzed using thematic analysis.Interviews with 11 IRB members revealed similar approaches to reviewing EFIC studies. Most use summaries of CC activities to determine community members’ attitudes; none (...) reported using specific criteria nor recalled any CC reviews that resulted in modifications to or denials of EFIC studies. Most interviewees thought metrics based on Community VOICES’s domains (feasibility, participant selection, quality of communication, community perceptions, investigator/IRB perceptions) would be helpful.IRB members had similar experiences and concerns about reviewing EFIC studies. Development of metrics to assess CC processes may be useful to IRBs reviewing EFIC studies. (shrink)

Serious games have become an important tool to train individuals in a range of different skills. Importantly, serious games or gamified scenarios allow for simulating realistic time-critical situations to train and also assess individual performance. In this context, determining the user’s cognitive load during training seems crucial for predicting performance and potential adaptation of the training environment to improve training effectiveness. Therefore, it is important to identify in-game metrics sensitive to users’ cognitive load. According to Barrouillets’ time-based resource-sharing (...) model, particularly relevant for measuring cognitive load in time-critical situations, cognitive load does not depend solely on the complexity of actions but also on temporal aspects of a given task. In this study, we applied this idea to the context of a serious game by proposing in-game metrics for workload prediction that reflect a relation between the time during which participants’ attention is captured and the total time available for the task at hand. We used an emergency simulation serious game requiring management of time-critical situations. Forty-seven participants completed the emergency simulation and rated their workload using the NASA-TLX questionnaire. Results indicated that the proposed in-game metrics yielded significant associations both with subjective workload measures as well as with gaming performance. Moreover, we observed that a prediction model based solely on data from the first minutes of the gameplay predicted overall gaming performance with a classification accuracy significantly above chance level and not significantly different from a model based on subjective workload ratings. These results imply that in-game metrics may qualify for a real-time adaptation of a game-based learning environment. (shrink)

We describe the design and evaluation of a sub-clinical digital assessment tool that integrates digital biomarkers of depression. Based on three standard cognitive tasks on which people with depression have been known to perform differently than a control group, we iteratively designed a digital assessment tool that could be deployed outside of laboratory contexts, in uncontrolled home environments on computer systems with widely varying system characteristics. We conducted two online studies, in which participants used the assessment (...) tool in their own homes, and completed subjective questionnaires including the Patient Health Questionnaire —a standard self-report tool for assessing depression in clinical contexts. In a first study, we demonstrate that each task can be used in isolation to significantly predict PHQ-9 scores. In a second study, we replicate these results and further demonstrate that when used in combination, behavioral metrics from the three tasks significantly predicted PHQ-9 scores, even when taking into account demographic factors known to influence depression such as age and gender. A multiple regression model explained 34.4% of variance in PHQ-9 scores with behavioral metrics from each task providing unique and significant contributions to the prediction. (shrink)

To assess ethics pedagogy in science and engineering, we developed a new tool called the Engineering and Science Issues Test (ESIT). ESIT measures moral judgment in a manner similar to the Defining Issues Test, second edition, but is built around technical dilemmas in science and engineering. We used a quasi-experimental approach with pre- and post-tests, and we compared the results to those of a control group with no overt ethics instruction. Our findings are that several (but not all) stand-alone classes (...) showed a significant improvement compared to the control group when the metric includes multiple stages of moral development. We also found that the written test had a higher response rate and sensitivity to pedagogy than the electronic version. We do not find significant differences on pre-test scores with respect to age, education level, gender or political leanings, but we do on whether subjects were native English speakers. We did not find significant differences on pre-test scores based on whether subjects had previous ethics instruction; this could suggest a lack of a long-term effect from the instruction. (shrink)

Figurative drawing is a skill that takes time to learn, and it evolves during different childhood phases that begin with scribbling and end with representational drawing. Between these phases, it is difficult to assess when and how children demonstrate intentions and representativeness in their drawings. The marks produced are increasingly goal-oriented and efficient as the child’s skills progress from scribbles to figurative drawings. Pre-figurative activities provide an opportunity to focus on drawing processes. We applied fourteen metrics to two different (...) datasets (_N_ = 65 and _N_ = 344) to better understand the intentional and representational processes behind drawing, and combined these metrics using principal component analysis (PCA) in different biologically significant dimensions. Three dimensions were identified: efficiency based on spatial metrics, diversity with color metrics, and temporal sequentiality. The metrics at play in each dimension are similar for both datasets, and PCA explains 77% of the variance in both datasets. Gender had no effect, but age influenced all three dimensions differently. These analyses for instance differentiate scribbles by children from those drawn by adults. The three dimensions highlighted by this study provide a better understanding of the emergence of intentions and representativeness in drawings. We discussed the perspectives of such findings in comparative psychology and evolutionary anthropology. (shrink)

Background and ObjectivesFocal cortical dysplasia is a type of malformations of cortical development and one of the leading causes of drug-resistant epilepsy. Postoperative results improve the diagnosis of lesions on structural MRIs. Advances in quantitative algorithms have increased the identification of FCD lesions. However, due to significant differences in size, shape, and location of the lesion in different patients and a big deal of time for the objective diagnosis of lesion as well as the dependence of individual interpretation, sensitive approaches (...) are required to address the challenge of lesion diagnosis. In this research, a FCD computer-aided diagnostic system to improve existing methods is presented.MethodsMagnetic resonance imaging data were collected from 58 participants. Morphological and intensity-based features were calculated for each cortical surface and inserted into an artificial neural network. Statistical examinations evaluated classifier efficiency.ResultsNeural network evaluation metrics—sensitivity, specificity, and accuracy—were 96.7, 100, and 98.6%, respectively. Furthermore, the accuracy of the classifier for the detection of the lobe and hemisphere of the brain, where the FCD lesion is located, was 84.2 and 77.3%, respectively.ConclusionAnalyzing surface-based features by automated machine learning can give a quantitative and objective diagnosis of FCD lesions in presurgical assessment and improve postsurgical outcomes. (shrink)

Background: Personality pathology does not have to be a contraindication to a bariatric surgery if a proper pre-surgical assessment is done. Indicating subgroups of patients with their specific needs could help tailor interventions and improve surgical treatment outcomes.Objectives: Using the Alternative DSM-5 model for personality disorders and the ICD-11 model for PDs to detect subgroups of patients with obesity based on a specific constellation of maladaptive personality traits and the level of overall personality impairment.Methods: 272 consecutively consented patients (...) who underwent a standard pre-surgical psychological assessment. The majority were women, age range was 22–79 years. Patients’ average body mass index was 43.95 kg/m2. All participants were administered the Personality Inventory for DSM-5 from which Level of Personality Functioning Scale-Self Report and Standardized Assessment of Severity of Personality Disorder scores were gained using the “crosswalk” for common metric for self-reported severity of personality disorder. The k-means clustering method was used to define specific subgroups of patients with obesity and replicated for equality testing to the samples of non-clinical respondents and psychiatric patients.Results: The cluster analysis detected specific groups in the sample of patients with obesity, which differed quantitatively from the samples of non-clinical respondents and psychiatric patients. A vast majority of patients with obesity showed above-average values in most of the PID-5 facets compared to the United States representative general community sample. In two out of the three clusters defined, patients demonstrated moderate to severe personality psychopathology within the Detachment and Negative Affectivity domains according to PID-5, which in one of the clusters corresponded to the mild overall impairment in both, LPFS-SR and SASPD. Moreover, higher levels of psychopathology prove to be associated with higher age and use of psychiatric medication.Conclusions: The dimensional DSM-5 and ICD-11 trait models are suitable procedures for defining specific “characters” of patients in a pre-bariatric setting. As such, they help to identify subgroups of patients with obesity who are different from general population and psychiatric patients. Implications for clinical practice and further research are discussed. (shrink)

Urban food sovereignty is a growing field of research and a site of struggle for food justice advocates, but it has gained less attention in low-income contexts, particularly in the Global South. Yet, with high rates of urbanization, and growing rates of urban poverty in many countries, urban food sovereignty, and the dietary, food systems and health aspirations of the urban poor should be taken seriously. In this paper, I explore the utility of a community-based tool for assessing food (...) sovereignty, and a case study of urban women at the Centre for Nutrition and Education for Women and Children (C-New-C) in Cap-Haitian, Haiti. Ultimately, the findings suggest that food sovereignty tools, assessments and metrics, when used in urban areas, can illuminate much that a food security assessment might overlook including, the importance of urban dietary aspirations, the value of traditional foods, the significance of land and gardening access to health and mental health, the impacts of gender on food access, and the possibilities for healthy urban food systems and communities. (shrink)

The aim of this research was to develop and validate the Questionnaire for Assessing Educational Podcasts, an instrument designed to gather students’ views about four dimensions of educational podcasts: access and use, design and structure, content adequacy, and value as an aid to learning. In study 1 we gathered validity evidence based on test content by asking a panel of experts to rate the clarity and relevance of items. Study 2 examined the psychometric properties of the QAEP, including confirmatory (...) factor analysis with cross-validation to test the factor structure of the questionnaire, as well as item and reliability analysis. The results from study 1 showed that the experts considered the items to be clearly worded and relevant in terms of their content. The results from study 2 showed a factor structure consistent with the underlying dimensions, as well as configural and metric invariance across groups. The item analysis and internal consistency for scores on each factor and for total scores were also satisfactory. The scores obtained on the QAEP provide teachers with direct student feedback and highlight those aspects that need to be enhanced in order to improve the teaching/learning process. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Close Enemies: The Relationship of Psychiatry and Psychology in the Assessment of Mental DisordersPhilippe Le Moigne, PhDAs Peter Zachar rightly points out in his comment, the assessment of mental disorders underwent new developments with the release of the Diagnostic and Statistical Manual of Mental Disorders-V in 2013 (American Psychiatric Association, 2013). Whereas in 1980, the manual had been thought of in a rigorously categorical way, on the (...) basis of distinct and closed syndromic entities, this new version advocated the development of a radically dimensionalist approach, with the diagnosis aiming this time to restore the whole symptomatological picture of the patient on a spectrum of ratings are supposed to cover the entirety of psychiatric semiology (American Psychiatric Association, 2013).For its part, my article aimed to show how standardized assessment, guided by the return of psychiatry’s authority and the assertion of its legitimacy in the diagnostic field, gave the ascendancy in the 1980s to the classificatory tradition and thus to the categorical model, to the detriment of personality approach and test psychology which offered a competing solution here. This perspective, historically dated, may have led the reader to think that there is an antagonism that cannot be overcome here. Rather, it should be seen as a form of structuring opposition, or even a “complementarity inscribed in rivalry,” between psychiatry and psychology, which is naturally susceptible to change over time.Indeed, the reversal operated by the Diagnostic and Statistical Manual of Mental Disorders-V is not total because it cannot be total. The defense of the dimensionalist approach, dear to the psychological testing tradition, was intended here to allow the identification of co-morbidities, present in most patients but poorly apprehended by the categorical approach, and to open up diagnosis to prevention by screening for minor manifestations of psychopathology when they suggest the [End Page 259] development of a manifest episode of the disease in the more or less long term. The risk here is that we end up with overdiagnosis and thus an over-representation of false positives (Wakefield, 2015). In other words, this type of approach does not escape the obligation to determine threshold scores, likely to indicate the lower limit of the disorder, and thus to resort to a categorical approach. There is a first affinity between the two perspectives which obeys, beyond the divergences in principle, a technical requirement.But the two approaches have even more in common when we look at their epistemological background: they share the idea that the representation and the intervention on mental disorder, to use Ian Hacking’s expression (1983), must be based on the model proposed by the physical and natural sciences. And as Greta Kaluzeviciute aptly points out in her comment, this orientation leads to the idea of measuring mental disorder in particular to conduct elaborate quantification operations to determine, for example, whether two entities are statistically distinct or whether a drug has a significant effect on the evolution of a pathology.This is in fact an old question that can be formulated as follows, to use the expression of the psychometrician Joel Michell (1999): are psychological dimensions metric? In other words, if we want to apply the language of numbers to mental disorders, they must share its properties, two of which seem essential: it must be possible to establish that each mental disorder, in the same way as qualities attributable to length for example, has a zero and can be divided by means of gradients (or degrees of intensity) whose interval is constant (Le Moigne, 2018; Michell, 1999). For lack of space, let us only address the second condition. This one has already been denied by many, starting with Bergson at the end of the nineteenth century (1889). He does not believe in the quantification of psychological expressions, starting with sensations. What is the common point, he wrote, between the feeling of a mild pain, as one feels a stiffness, and the sensation of a needle being hammered into your flesh? For him, one and the other of these intensities are incomparable, so that they do not belong to a similar register of sensations that one could stage from a 0 toward a... (shrink)

Impaired neurodevelopmental outcome, in particular cognitive impairment, after neonatal hypoxic-ischemic encephalopathy is a major concern for parents, clinicians, and society. This study aims to investigate the potential benefits of using advanced quantitative electroencephalography analysis for early prediction of cognitive outcomes, assessed here at 2 years of age. EEG data were recorded within the first week after birth from a cohort of twenty infants with neonatal hypoxic-ischemic encephalopathy. A proposed regression framework was based on two different sets of features, namely (...) graph-theoretical features derived from the weighted phase-lag index and entropies metrics represented by sample entropy, permutation entropy, and spectral entropy. Both sets of features were calculated within the noise-assisted multivariate empirical mode decomposition domain. Correlation analysis showed a significant association in the delta band between the proposed features, graph attributes and entropy features from the neonatal EEG data and the cognitive development at age two years. These features were used to train and test the tree ensemble regression models. The highest prediction performance was reached to 14.27 root mean square error, 12.07 mean absolute error, and 0.45 R-squared using the entropy features with a boosted tree regression model. Thus, the results demonstrate that the proposed qEEG features show the state of brain function at an early stage; hence, they could serve as predictive biomarkers of later cognitive impairment, which could facilitate identifying those who might benefit from early targeted intervention. (shrink)

The productivity of researchers and the impact of the work they do are a preoccupation of universities, research funding agencies, and sometimes even researchers themselves. The h-index is the most popular of different metrics to measure these activities. This research deals with presenting a practical approach to model the h-index based on the total number of citations and the duration from the publishing of the first article. To determine the effect of every factor on h, we applied a (...) set of simple nonlinear regression. The results indicated that both NC and D1 had a significant effect on h. The determination of coefficient for these equations to estimate the h-index was 93.4% and 39.8%, respectively, which verified that the model based on NC had a better fit. Then, to record the simultaneous effects of NC and D1 on h, multiple nonlinear regression was applied. The results indicated that NC and D1 had a significant effect on h. Also, the determination of coefficient for this equation to estimate h was 93.6%. Finally, to model and estimate the h-index, as a function of NC and D1, multiple nonlinear quartile regression was used. The goodness of the fitted model was also assessed. (shrink)

In real life, multiple network public opinion emergencies may break out in a certain place at the same time. So, it is necessary to invite emergency decision experts in multiple fields for timely evaluating the comprehensive crisis of the online public opinion, and then limited emergency resources can be utilized to give priority to respond to the one with the highest crisis. Due to the complexity of network public opinion emergencies and the limited cognition of experts, most of the decision (...) problems for evaluating the network public opinion emergencies are highly uncertain. Also, prior to the selection of the highest crisis, it is preferable that experts reach a high degree of consensus among their assessments or opinions. To address such problems, this paper presents a novel adaptive consensus reaching model for multiattribute group decision making with probabilistic linguistic decision matrices. First, to quantify the difference between any two probabilistic linguistic term sets accurately and efficiently, we define a novel distance measure between PLTSs based on the Wasserstein metric. Then, by integrating the defined PLTSs-based Wasserstein distance measure into the classical CCSD method, we construct an optimization model for objectively determining attribute weights. Subsequently, we develop the individual cumulative consensus contribution measure and the group consensus measure, respectively, following which is to present an integrated consensus improving strategy that considers both weight-updating and assessment-adjusting. Finally, the feasibility and the applicability of the proposed approach are illustrated via a real evaluation of network public opinion emergencies. Through comparing with existing probabilistic linguistic MAGDM approaches, the proposed approach offers the advantages in terms of the accurate measurement of information difference and the integrated improvement of consensus efficiency. (shrink)

Artificial intelligence (AI) governance and auditing promise to bridge the gap between AI ethics principles and the responsible use of AI systems, but they require assessment mechanisms and metrics. Effective AI governance is not only about legal compliance; organizations can strive to go beyond legal requirements by proactively considering the risks inherent in their AI systems. In the past decade, investors have become increasingly active in advancing corporate social responsibility and sustainability practices. Including nonfinancial information related to environmental, (...) social, and governance (ESG) issues in investment analyses has become mainstream practice among investors. However, the AI auditing literature is mostly silent on the role of investors. The current study addresses two research questions: (1) how companies’ responsible use of AI is included in ESG investment analyses and (2) what connections can be found between principles of responsible AI and ESG ranking criteria. We conducted a series of expert interviews and analyzed the data using thematic analysis. Awareness of AI issues, measuring AI impacts, and governing AI processes emerged as the three main themes in the analysis. The findings indicate that AI is still a relatively unknown topic for investors, and taking the responsible use of AI into account in ESG analyses is not an established practice. However, AI is recognized as a potentially material issue for various industries and companies, indicating that its incorporation into ESG evaluations may be justified. There is a need for standardized metrics for AI responsibility, while critical bottlenecks and asymmetrical knowledge relations must be tackled. (shrink)

The debate on ideal theory focuses mainly on whether it can provide a long-term target and a metric for assessing the justice of different institutional arrangements in non-ideal theory. Both critics and defenders of ideal theory typically overlook the role it plays in a model of fairness that can restrict the range of permissible arrangements under non-ideal conditions. In this paper, I explain ideal theory’s fairness role and its part in ensuring an institutional structure that benefits everyone in a society. (...) Critics of ideal theory therefore cannot reject it as simply useless. But I consider how they can question the attractiveness of an ideal-theory-based fairness constraint on non-ideal theory. (shrink)

This study explores the phenomenon of artificial intelligence (AI) aversion within the context of public policy, building on prior research on algorithmic aversion. I aim to establish a clear conceptual distinction between algorithms and AI in the public’s perception and develop a robust metric for assessing AI aversion. Utilizing a national survey, I employed affective imagery testing to compare Americans emotional responses towards AI, algorithms, and advanced technology. The findings reveal that AI elicits significantly more negative emotional responses than the (...) other two, indicating its unique position in public perception. I then construct the Artificial Intelligence Aversion Index (AIAI) based on responses to policy-related vignettes. Regression analyses showed a strong negative relationship between the AIAI and public support for both current and future AI applications within public policy, with aversion more pronounced towards potential future uses. These insights underscore the importance of understanding public sentiment towards AI to inform policymaking as well as helping to establish a framework by which to evaluate aversion levels. (shrink)

Background Informed consent confers upon participants the right to decline or accept participation in a study in equal measure. Consent declines can be used as a potential metric to assess the adequacy of the informed consent process. Limited literature is available on the reporting of consent declines in Clinical Research. We evaluated randomized controlled trials published in three high-impact factor journals for consent declines (four-year period) to assess the extent of exertion of autonomy by research participants. Methods CONSORT flow charts (...) in RCTs published in New England Journal of Medicine (NEJM), The Lancet and Journal of the American Medical Association (JAMA) between 2012 and 2015 were evaluated. The total and per journal proportion of RCTs reporting consent declines was calculated. Percentage of consent declines was also compared based on number of centers (single centric vs. multicentric) and type of setting (developed vs. developing). Between-groups analysis was done using Chi Square, Chi Square for trend and a Crude odds ratio [95% C.I] (cOR) were calculated. Results Only 3.7% (1103/29,825) of the published articles were RCTs. Of the 1103 RCTs, only 532 (48.2%) reported consent declines. JAMA had the highest RCTs reporting consent declines (171/267.64%), the Lancet [129,150/781,924, 16.5%] had the most participants declining consent. Difference was significant using Chi square for linear trend [ p < 0.00001]. Consent decline rate was higher in developed countries relative to developing countries [cOR(95% C.I): 2.27(2.23,2.32), p < 0.000001] and in single centric versus multicentric studies [cOR (95% C.I): 3.63(3.56,3.71), p < 0.0000001]. Conclusion Both authors and journal editors need to ensure better reporting of consent declines. (shrink)

Anderson and Dyck claim that the current trend of almost exclusively using citation-based evaluative metrics to assess the research output of scholars is unsound. I agree with them in this, but I feel that, for practical reasons, this system will not disappear in the near future, so we must concentrate on making it fairer. Both commentators doubt whether numerically expressing each contributor's relative contribution is feasible. I admit that an important precondition for this task is the possibility of (...) an informed, democratic debate among equals about the relative contribution of each contributor to the article. Mechanisms should be established to protect vulnerable researchers in the academic field in the same way as safeguards exist today to protect vulnerable research participants. Theoretically, however, I think that the fair allocation of authorship credit is possible, and much of this task is already being performed routinely when contributors determine the order of their names in the byline, being well aware of the widespread assumption that this order mostly mirrors the order of their relative contributions. All they would have to do as an additional task is to express this order in numbers. If they cannot reach a consensus, they could always choose not to express their relative contribution in numbers, in which case the presumption would be that they contributed equally. My proposal could, at best, make the system fairer and, at worst, not reduce the options that evaluators already have. (shrink)

Traditional methods for deriving property-based representations of concepts from text have focused on either extracting only a subset of possible relation types, such as hyponymy/hypernymy (e.g., car is-a vehicle) or meronymy/metonymy (e.g., car has wheels), or unspecified relations (e.g., car—petrol). We propose a system for the challenging task of automatic, large-scale acquisition of unconstrained, human-like property norms from large text corpora, and discuss the theoretical implications of such a system. We employ syntactic, semantic, and encyclopedic information to guide our (...) extraction, yielding concept-relation-feature triples (e.g., car be fast, car require petrol, car cause pollution), which approximate property-based conceptual representations. Our novel method extracts candidate triples from parsed corpora (Wikipedia and the British National Corpus) using syntactically and grammatically motivated rules, then reweights triples with a linear combination of their frequency and four statistical metrics. We assess our system output in three ways: lexical comparison with norms derived from human-generated property norm data, direct evaluation by four human judges, and a semantic distance comparison with both WordNet similarity data and human-judged concept similarity ratings. Our system offers a viable and performant method of plausible triple extraction: Our lexical comparison shows comparable performance to the current state-of-the-art, while subsequent evaluations exhibit the human-like character of our generated properties. (shrink)

In this pilot study, deliberate practice and competence-based assessment were incorporated into a first-year graduate course on professional issues and ethics. Students responded to challenging simulations of basic ethical situations in therapy before and after the course. Aspects of deliberate practice were incorporated into the course. Student self-report ratings and independent performance ratings blind to timing found improvements in students’ ability to manage basic ethical situations in practice. Pilot evidence suggests competence-based assessment and training have potential (...) use for training students preparing to engage in their first therapy practicum experiences. Additional controlled studies are needed to examine the benefits of competence-based assessment and training. (shrink)

This article critically assesses contemporary empirical educational research, directing attention toward overlooked facets of pedagogical practice. Here, Agnes Pfrang and Daniel Castner raise questions about predominant psychological approaches to empirical educational research, instead advocating for a holistic viewpoint that encompasses the subtleties of educational situations and experiences. They highlight the learning atmosphere and pedagogical relationships as crucial dimensions often neglected by researchers. By delving into the historical evolution of the relationship between educational research and empirical pedagogy, the article underscores the (...) ramifications of an increasing emphasis on evidence-based approaches. It contends that conventional control measures, focused on performance metrics and rankings, inadequately encapsulate the essence of pedagogy. Pfrang and Castner call for a paradigm shift, urging a more comprehensive and context-sensitive framework in both educational research and practice. This article serves as a rallying cry for a nuanced understanding that embraces the multifaceted nature of educational interactions, advocating for a move away from traditional metrics and toward more inclusive perspectives. (shrink)

Objective: Traditionally, asymmetric spatial processing has been assessed with paper-and-pencil tasks, but growing evidence indicates that computer-based methods are a more sensitive assessment modality. It is not known, however, whether simply converting well-established paper-and-pencil methods into a digital format is the best option. The aim of the present study was to compare sensitivity in detecting contralesional omissions of two different computer-based methods: a “digitally converted” cancellation task was compared with a computer-based Visual and Auditory dual-tasking approach, (...) which has already proved to be very sensitive.Methods: Participants included 40 patients with chronic unilateral stroke in either the right hemisphere or the left hemisphere and 20 age-matched healthy controls. The cancellation task was implemented on a very large format or in a smaller paper-and-pencil version. The computer-based dual-tasks were implemented on a 15′′ monitor and required the detection of unilateral and bilateral briefly presented lateralized targets.Results: Neither version of the cancellation task was able to show spatial bias in RH patients. In contrast, in the Visual dual-task RH patients missed significantly more left-sided targets than controls in both unilateral and bilateral trials. They also missed significantly more left-sided than right-sided targets only in the bilateral trials of the Auditory dual-task.Conclusion: The dual-task setting outperforms the cancellation task approach even when the latter is implemented on a screen. Attentionally demanding methods are useful for revealing mild forms of contralesional visuospatial deficits. (shrink)

This research project explores the effectiveness of simulation-based assessments (SBAs) as an alternative to practical assessments for event management (EM) students, considering the 2020 global pandemic's impact on higher education. SBAs mimic real-world scenarios, enabling innovative and authentic ways for assessors to determine how well candidates can put their knowledge, skills, and talents to use. For example, they give students an opportunity to instil and reinforce ethical principles and values essential for success in the workplace. The study aims to (...) investigate the relationship between SBAs and graduate attributes, via a qualitative approach, to investigate and analyse students' lived experiences. Ten final-year EM students were selected purposively, with inclusion criteria. Semi-structured interviews were conducted using an original set of selected questions, and thematic analysis was used for data analysis. The study's findings suggest that if used correctly, SBAs can be an effective tool for developing students' graduate attributes, preparing them for employment. The article concludes on how to maximise the value of SBAs in preparing students for the workforce, especially when practical assessments are not feasible. (shrink)

Based on practical and thoroughly researched information, this book provides a much-needed framework for understanding the causes and developing solutions to the problem of sexual aggression.

Reports of the Intergovernmental Panel on Climate Change (IPCC) employ an evolving framework of calibrated language for assessing and communicating degrees of certainty in findings. A persistent challenge for this framework has been ambiguity in the relationship between multiple degree-of-certainty metrics. We aim to clarify the relationship between the likelihood and confidence metrics used in the Fifth Assessment Report (2013), with benefits for mathematical consistency among multiple findings and for usability in downstream modeling and decision analysis. We (...) discuss how our proposal meshes with current and proposed practice in IPCC uncertainty assessment. (shrink)

As a way of human-computer interaction, game-based assessment is more suitable for young children because it is situational, interesting, and effective. National identity is an important factor affecting the overall development of young children and the future development of a country, which has attracted extensive attention from researchers. Nevertheless, the assessment of young children's national identity is still based on traditional evaluation, including questionnaires and interviews, which have the limitations of being inaccurate, dull, and time-consuming. To (...) understand the characteristics of children's national identity, it is necessary to use scientific and interactive assessment methods. The present study investigated whether the game-based assessment we developed specifically would be an appropriate tool to measure young children's national identity. The results show that the game-based assessment had good item discrimination. Exploratory factor analysis demonstrated the game covered three aspects: national cognition mastery, national emotion engagement, and national behavior tendency. The confirmatory factor analysis suggested that the model with three factors fit the data well. The internal consistency, the split-half reliability, and the test-retest reliability meet standards. Overall, the results indicated that this game can be successfully used to assess young children's national identity with acceptable validity and reliability. Our study provides strong evidence for the use of human-computer interaction in child measurement. These findings are the first to demonstrate the promise of game-based assessment in assessing children's national identity reliably and effectively. (shrink)

Systems medicine, which is based on computational modelling of biological systems, is emerging as an increasingly prominent part of the personalized medicine movement. It is often promoted as ‘P4 medicine’. In this article, we test promises made by some of its proponents that systems medicine will be able to develop a scientific, quantitative metric for wellness that will eliminate the purported vagueness, ambiguity, and incompleteness—that is, normativity—of previous health definitions. We do so by examining the most concrete and relevant (...) evidence for such a metric available: a patent that describes a systems medicine method for assessing health and disease. We find that although systems medicine is promoted as heralding an era of transformative scientific objectivity, its definition of health seems at present still normatively based. As such, we argue that it will be open to influence from various stakeholders and that its purported objectivity may conceal important scientific, philosophical, and political issues. We also argue that this is an example of a general trend within biomedicine to create overly hopeful visions and expectations for the future. (shrink)

We introduce a new metric for interdisciplinarity, based on co-author publication history. A published article that has co-authors with quite different publication histories can be deemed relatively “interdisciplinary,” in that the article reflects a convergence of previous research in distinct sets of publication outlets. In recent work, we have shown that this interdisciplinarity metric can predict citations. Here, we show that the journal Cognitive Science tends to contain collaborations that are relatively high on this interdisciplinarity metric, at about the (...) 80th percentile of all journals across both social and natural sciences. Following on Goldstone and Leydesdorff, we describe how scientometric tools provide a valuable means of assessing the role of cognitive science in broader scientific work, and also as a tool to investigate teamwork and distributed cognition. We describe how data-driven metrics of this kind may facilitate this exploration without relying upon rapidly changing discipline and topic keywords associated with publications. (shrink)

Technology-based assessment offers unique possibilities for collecting data about students’ cognitive development and using this data to provide students and teachers with feedback to improve learning. The aim of this study was to show how the psychological dimension of learning can be assessed in everyday educational practice through technology-based assessment in reading, mathematics and science. We analyzed three related aspects of the assessments: cognitive development, gender differences and vertical scaling. The sample for the study was drawn (...) from primary school students in Grades 1–8 (aged 7 to 14) in Hungary. There were 1500 to 2000 students in each grade cohort. Online tests were constructed from 1638 items from the reading, mathematics and science domains in the eDia system. The results confirmed empirically that the disciplinary, application and psychological dimensions of learning can be empirically distinguished. Students’ cognitive development was the most steady (and effective) in mathematics where the greatest development happened in the first years of schooling. Path models suggested that the psychological dimension of learning can be predicted at a moderate level based on students’ level of school knowledge consisting of the disciplinary and application dimensions of learning as latent constructs. The predictive power was almost the same in both of the dimensions. Generally, girls developed faster in the psychological dimension of reading, mathematics and science learning; however, the size of gender differences varied by age and domain. This study (1) provides evidence that the psychological dimension of learning can be made visible even in an educational context and (2) highlights the importance of the explicit development of the psychological dimension of learning during school lessons, and (3) shows that there are gender differences in the developmental level of the psychological dimension of learning in favour of girls, but it varies by grade and domain. (shrink)

Identifying sporting talent remains a difficult task due to the complex nature of sport. Technical skill assessments are used throughout the talent pathway to monitor athletes in an attempt to more effectively predict future performance. These assessments however, largely focus on the isolated execution of key skills devoid of any game context. When assessments are representative of match-play and applied in a setting where all four components of competition (i.e., technical, tactical, physiological and psychological) are assessed within an integrated approach, (...) prediction of talent is more likely to be successful. This article explores the current talent identification technical skill assessments, with a particular focus on Australian Football, and proposes a 5-level performance assessment model for athlete assessment. The model separates technical game skill on a continuum from Level-1 (i.e., laboratory analysis) to Level-5 (i.e., match-play). These levels, using the assumptions of the expert performance model and representative learning design theory, incorporate a step-wise progression of performance demands to more closely represent match-play conditions. The proposed model will provide researchers and practitioners with a structured framework to consider when assessing, or developing, new assessments of technical game-based skill. (shrink)

This article deals with a modern disease of academic science that consists of an enormous increase in the number of scientific publications without a corresponding advance of knowledge. Findings are sliced as thin as salami and submitted to different journals to produce more papers. If we consider academic papers as a kind of scientific ‘currency’ that is backed by gold bullion in the central bank of ‘true’ science, then we are witnessing an article-inflation phenomenon, a scientometric bubble that is most (...) harmful for science and promotes an unethical and antiscientific culture among researchers. The main problem behind the scenes is that the impact factor is used as a proxy for quality. Therefore, not only for convenience, but also based on ethical principles of scientific research, we adhere to the San Francisco Declaration on Research Assessment when it emphasizes “the need to eliminate the use of journal-based metrics in funding, appointment and promotion considerations; and the need to assess research on its own merits rather on the journal in which the research is published”. Our message is mainly addressed to the funding agencies and universities that award tenures or grants and manage research programmes, especially in developing countries. The message is also addressed to well-established scientists who have the power to change things when they participate in committees for grants and jobs. (shrink)