Meta-Analysis and Systematic Review
Meta-analysis
by Thom Baguley
Baguley, T. (2012). Meta-analysis. Online Supplement 1 to Serious stats: A guide to advanced statistics for the behavioral sciences. Basingsoke: Palgrave. Retrieved from: http://www.palgrave.com/psychology/baguley/students/supplements.html
Companion blog:
http://seriousstats.wordpress.com/
An Approach to Comparing Nations for Inclusion of Studies in Health-based Systematic Literature Reviews
Background: To develop and demonstrate a systematic approach for comparing nations, for the purpose of deciding... more Background: To develop and demonstrate a systematic approach for comparing nations, for the purpose of deciding whether to include or exclude studies in a systematic review of a health research question pertinent to the Canadian population. Results: a template of nine criteria was developed, including both sociodemographic and systemic indicators, and was applied to 68 jurisdictions, of which 19 were deemed sufficiently comparable to Canada to be included in the review. Conclusions: Subsequent systematic reviews, regardless of which nation is ultimately the reference population, should employ a similar process, with indicators and characteristics specific to the research questions, to ensure that political, economic, historical or ethnic biases are not influencing the selection or rejection of relevant papers.
Download (.pdf) Probiotics for the Prevention and Treatment of Antibiotic-Associated Diarrhea
Susanne Hempel, PhD; Sydne J. Newberry, PhD; Alicia R. Maher, MD; Zhen Wang, PhD; Jeremy N. V. Miles, PhD; Roberta Shanman, MS; Breanne Johnsen, BS; Paul G. Shekelle, MD, PhD; JAMA. 2012;307(18):1959-1969. doi: 10.1001/jama.2012.3507
Context Probiotics are live microorganisms intended to confer a health benefit when consumed. One condition for which... more
Context Probiotics are live microorganisms intended to confer a health benefit when consumed. One condition for which probiotics have been advocated is the diarrhea that is a common adverse effect of antibiotic use.
Objective To evaluate the evidence for probiotic use in the prevention and treatment of antibiotic-associated diarrhea (AAD).
Data Sources Twelve electronic databases were searched (DARE, Cochrane Library of Systematic Reviews, CENTRAL, PubMed, EMBASE, CINAHL, AMED, MANTIS, TOXLINE, ToxFILE, NTIS, and AGRICOLA) and references of included studies and reviews were screened from database inception to February 2012, without language restriction.
Study Selection Two independent reviewers identified parallel randomized controlled trials (RCTs) of probiotics (Lactobacillus, Bifidobacterium, Saccharomyces, Streptococcus, Enterococcus, and/or Bacillus) for the prevention or treatment of AAD.
Data Extraction Two independent reviewers extracted the data and assessed trial quality.
Results A total of 82 RCTs met inclusion criteria. The majority used Lactobacillus -based interventions alone or in combination with other genera; strains were poorly documented. The pooled relative risk in a DerSimonian-Laird random-effects meta-analysis of 63 RCTs, which included 11 811 participants, indicated a statistically significant association of probiotic administration with reduction in AAD (relative risk, 0.58; 95% CI, 0.50 to 0.68; P < .001; I2, 54%; [risk difference, −0.07; 95% CI, −0.10 to −0.05], [number needed to treat, 13; 95% CI, 10.3 to 19.1]) in trials reporting on the number of patients with AAD. This result was relatively insensitive to numerous subgroup analyses. However, there exists significant heterogeneity in pooled results and the evidence is insufficient to determine whether this association varies systematically by population, antibiotic characteristic, or probiotic preparation.
Conclusions The pooled evidence suggests that probiotics are associated with a reduction in AAD. More research is needed to determine which probiotics are associated with the greatest efficacy and for which patients receiving which specific antibiotics.
THE TWO PRESIDENCIES AN UNFINISHED PROJECT: A THEORETICAL, METHODOLOGICAL, EMPIRICAL & NORMATIVE META-ANALYSIS
Paper prepared for the Southwestern Social Science Association 2006 Annual Meeting at San Antonio, TX April 11-15, 2006
In this paper, I present evidence from a meta-analysis of one of the most dominant theoretical perspectives in... more
In this paper, I present evidence from a meta-analysis of one of the most dominant theoretical perspectives in executive-legislative policy making relations—Aaron Wildavsky’s (1966) two presidencies thesis. Wildavsky’s work suggested that no less than two policy making presidencies existed within a single president’s relationship with the Congress (1966). Furthermore, that while the president “dominated” the construction of policies in foreign affairs vis-à-vis the Congress; he was impeded in that effort in the realm of domestic politics (1966).
After Wildavsky’s work was published no less than an entire school of academic thought grew up around the idea and inherent possibilities of a “two presidencies thesis” (Shull 1991). It is my intention in this analysis to examine that body of literature and report on its general conclusions relative to the existence of and subsequent evolution of the two presidencies as an intellectual endeavor. My findings indicate that the two presidencies is subject to examination within institutional, methodological and partisan terms and from that an adequate theoretical, empirical and normative critique can be developed which is the finished project of this paper.
As an empirical theory, the two presidencies is rather strongly support with a support rate among two presidency researchers of 62.5%. The strongest level for the explanatory existence of the two presidencies comes from Wildavsky’s classic institutional (Wildavsky 1966) version and the strongest reason for rejection of the two presidencies comes from its notion as a “cultural phenomenon” (Peppers 1975). Despite wide spread discussion within the literature there is little evidence to support a strictly partisan version of the thesis probably due to its lack of clear association between itself and either support or refutation of the thesis. (see Edwards 1986). Methodologically, the two presidencies has some association between the employment of an aggregate level of analysis and the institutional explanation, however, while some qualitative evidence exists for such a relationship between the employment of an individual level of analysis and the partisan version of the two presidencies no such relationship is corroborated quantitatively. Also, the cultural version of the theory is associated at least through the qualitative meta-analysis with the employment of qualitative methods; this relationship does not appear in the quantitative portion of the meta-analysis. Finally, the lack of normative study either as a critique of executive-legislative relations or as a reflexive commentary on the body of research itself holds the two presidencies back from fulfilling a broader promise regarding American political analysis.
Lastly, the need for re-theorization is necessitated by the conclusions of this meta-analysis because the two presidencies as it currently exists is not developed enough to truly get at the “heart of the executive-legislative policy making divide.” A more nuanced approach is needed to provide greater empirical “fit,” provide stronger explanative/predictive “power,” and establish a normative “critique” as to the appropriateness of presidential prerogatives and congressional involvement in foreign and domestic policy making.
6 views
Seen by:
Abstract, Dedication, and Acknowledgments for the Hobbs (2011) dissertation published by SAS.
The Hobbs (2011) doctoral study is published in the ProQuest Dissertations and These database, UMI No. 3484309
The purpose of the qualitative research was to assess models of education developed for the study to investigate how... more The purpose of the qualitative research was to assess models of education developed for the study to investigate how and when to incorporate second and third languages into the curriculum to improve language acquisition. Research indicates that L3 enhances and reinforces L2 and L1. The stratified systematic grounded theory study explored the perspectives of neurolinguists, psycholinguists, sociolinguists, and interdisciplinary education researchers to derive variables for constructing a new model of education. The outcome of the Internet survey revealed that 100% of the participants agreed that education must change and that teacher training must improve. Variables from the cross-disciplinary data contributed to the construction of an integrated model of multilingual education consisting of four primary models and other models to serve as tools for designing curriculum, instruction, and assessment as well as determining demographics and student meta-analysis of language abilities and storage in the brain. The first model emerged from the data to offer multilingual principles of education. The other primary models are macro, meso, and micro models. The macro model represents schools, instruction, assessment, and the curriculum cycle. The meso model depicts the developmental domains of the individual learner and includes a cyclical equation. The micro model delineates multilingual processing in the brain based on neurolinguistic research, variables from the current study, and Kees de Bot's bilingual adaptation of Levelt's language processing model. Recommendations include the incorporation of notional-functional pragmatic-aesthetic concepts as depicted in the models developed for the study and enhanced by input from published researchers with unique language and research repertoires who were located on four continents.
27 views
Seen by: Anaesthesia for evacuation of incomplete miscarriage
Calvache JA, Delgado-Noguera MF, Lesaffre E, Stolker RJ.
Cochrane Systematic Review, April 2012
http://www.ncbi.nlm.nih.gov/pubmed/22513963#
Netherlands Institute for Health Sciences (NL), Department of Anesthesiology (COL), Erasmus University Rotterdam (NL), University of Cauca (COL), Educational Support Center (DCO) Room Ee308, PO Box 2040, Rotterdam, Netherlands, 3000 CA.
Abstract
BACKGROUND:
An incomplete miscarriage occurs when all the products of conception are not expelled... more
Abstract
BACKGROUND:
An incomplete miscarriage occurs when all the products of conception are not expelled through the cervix. Curettage or vacuum aspiration have been used to remove retained tissues. The anaesthetic techniques used to facilitate this procedure have not been systematically evaluated in order to determine which provide better outcomes to the patients.
OBJECTIVES:
To assess the effects of general anaesthesia, sedation or analgesia, regional or paracervical block anaesthetic techniques, or differing regimens of these, for surgical evacuation of incomplete miscarriage.
SEARCH METHODS:
We searched the Cochrane Pregnancy and Childbirth Group's Trials Register (23 January 2012), CENTRAL (The Cochrane Library 2012, Issue 1), PubMed (1966 to 23 January 2012), EMBASE (1974 to 23 January 2012), CINAHL (1982 to 23 January 2012), LILACS (1982 to 23 January 2012) and reference lists of retrieved studies.
SELECTION CRITERIA:
All published and unpublished randomised controlled trials (RCTs) or cluster-RCTs comparing the use of any anaesthetic technique (defined by authors as general anaesthesia, sedation/analgesia, regional or paracervical local block (PCB) procedures) to perform surgical evacuation of an incomplete miscarriage. We excluded quasi-randomised trials and studies that were only available as abstracts.
DATA COLLECTION AND ANALYSIS:
Two review authors independently assessed studies for inclusion and assessed risk of bias. Data were independently extracted and checked for accuracy.
MAIN RESULTS:
We included seven trials involving 800 women. The comparisons revealed a very high clinical heterogeneity. As a result of the heterogeneity in the randomisation unit, we did not combine trials but reported the individual trial results in the 'Data and analysis' section and in the text. Half of trials have unclear or high risk of bias in several domains.We did not find any trial reporting data about maternal mortality. In terms of postoperative pain, PCB does not improve the control of postoperative pain when it is compared against sedation/analgesia or versus no anaesthesia/no analgesia. In the comparison of PCB with lidocaine versus PCB with saline solution, significant differences favouring the group with lidocaine were found in one trial (moderate or severe postoperative pain) (risk ratio (RR) 0.32; 95% confidence interval (CI) 0.18 to 0.59).When opioids were used, postoperative nausea and vomiting was more frequent in two trials comparing those versus PCB. In terms of requirement of blood transfusion, two trials showed conflicting results.
AUTHORS' CONCLUSIONS:
Particular considerations that influence the choice of anaesthesia for this procedure such as availability, effectiveness, safety, side effects, practitioner's choice, costs and woman's preferences of each technique should continue to be used until more evidence supporting the use of one technique or another.
Exercise intervention and inflammatory markers in coronary artery disease: a meta-analysis
Swardfager W, Herrmann N, Cornish S, Mazereeuw G, Marzolini S, Sham L, Lanctôt KL. Am Heart J 2012;163:666-676.e3
Background: Inflammatory activity plays a role in the development and progression of coronary artery disease (CAD),... more
Background: Inflammatory activity plays a role in the development and progression of coronary artery disease (CAD), and exercise confers survival benefit. We performed a meta-analysis of changes in inflammatory biomarkers over the course of exercise interventions in patients with CAD.
Methods: We searched MEDLINE, Embase, the Cochrane Collaboration, AMED, and CINAHL for studies reporting peripheral inflammatory biomarker concentrations before and after exercise interventions of ≥2 weeks in patients with CAD. Data were summarized using standard mean differences (SMD) and 95% CIs.
Results: Twenty-three studies were included. Concentrations of C-reactive protein (CRP; SMD −0.345, 95% CI −0.444 to −0.246, n = 1,466, P < .001), interleukin 6 (SMD −0.546, 95% CI −0.739 to −0.353, n = 280, P < .001), fibrinogen (SMD −0.638, 95% CI −0.953 to −0.323, n = 247, P < .001), and vascular cell adhesion molecule 1 (SMD −0.413, 95% CI −0.778 to −0.048, n = 187, P = .027) were lower postintervention. Higher total cholesterol (B = −0.328, 95% CI −0.612 to −0.043, P = .026) and higher total/high-density lipoprotein cholesterol ratios (B = −0.250, 95% CI −0.425 to −0.076, P = .008) at baseline were associated with greater reductions in CRP. In controlled studies, follow-up concentrations of CRP (SMD −0.500, 95% CI −0.844 to −0.157, nexercise/control = 485/284, P = .004), and fibrinogen (SMD −0.544, 95% CI −1.058 to −0.030, nexercise/control = 148/100, P = .038) were lower in subjects who exercised compared with controls.
Conclusion: Exercise training is associated with reduced inflammatory activity in patients with CAD. C-reactive protein and fibrinogen have provided the strongest evidence. Higher baseline CRP and adverse baseline lipid profiles predicted greater reductions in CRP.
Analysis of the reporting of search strategies in Cochrane systematic reviews
by Daphne Plaut
Analysis of the reporting of search strategies in Cochrane systematic reviews.
Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE.
J Med Libr Assoc. 2009 Jan;97(1):21-9.
PMID: 19158999
Individual patient data meta-analysis of survival data using Poisson regression models
Crowther MJ, Riley RD, Staessen JA, Wang J, Gueyffier F and Lambert PC. BMC Medical Research Methodology 2012;12(34).
Background
An Individual Patient Data (IPD) meta-analysis is often considered the gold-standard for... more
Background
An Individual Patient Data (IPD) meta-analysis is often considered the gold-standard for synthesising survival data from clinical trials. An IPD meta-analysis can be achieved by either a two-stage or a one-stage approach, depending on whether the trials are analysed separately or simultaneously. A range of one-stage hierarchical Cox models have been previously proposed, but these are known to be computationally intensive and are not currently available in all standard statistical software. We describe an alternative approach using Poisson based Generalised Linear Models (GLMs).
Methods
We illustrate, through application and simulation, the Poisson approach both classically and in a Bayesian framework, in two-stage and one-stage approaches. We outline the benefits of our one-stage approach through extension to modelling treatment-covariate interactions and non-proportional hazards. Ten trials of hypertension treatment, with all-cause death the outcome of interest, are used to apply and assess the approach.
Results
We show that the Poisson approach obtains almost identical estimates to the Cox model, is additionally computationally efficient and directly estimates the baseline hazard. Some downward bias is observed in classical estimates of the heterogeneity in the treatment effect, with improved performance from the Bayesian approach.
Conclusion
Our approach provides a highly flexible and computationally efficient framework, available in all standard statistical software, to the investigation of not only heterogeneity, but the presence of non-proportional hazards and treatment effect modifiers.
Beyond FMEA: The Structured What-If Technique (SWIFT)
by Alan Card
Card AJ, Ward JR, Clarkson PJ. Beyond FMEA: The Structured What-If Technique (SWIFT). Journal of Healthcare Risk Management. 2012;31(4):23-9.
If you would like a copy of the final published version of this paper, and do not have access to the Journal of Healthcare Risk Management, feel free to write me at: alan.j.card [at] gmail.com.
Although it is probably the best-known Prospective Hazard Analysis (PHA) tool, Failure Mode and Effects Analysis... more
Although it is probably the best-known Prospective Hazard Analysis (PHA) tool, Failure Mode and Effects Analysis (FMEA) is far from the only option available. This paper introduces one of the alternatives: The Structured What-If Technique (SWIFT). SWIFT is a flexible, high-level risk identification technique that can be used on a standalone basis, or as part of a staged approach to make more efficient use of bottom-up methods like FMEA.
In this paper we describe the method, assess the evidence related to its use in healthcare using a systematic literature review, and suggest ways in which it could be better adapted for use in the healthcare industry. Based on the limited
evidence available, it appears that healthcare workers find it easy to learn, easy to use, and credible. Especially when used as part of a staged approach, SWIFT appears capable of playing a useful role as component of the PHA
armamentarium.
Technological pedagogical content knowledge – a review of the literature
by Jo Tondeur
Joke Voogt, Petra Fisser, Natalie Pareja Roblin, Jo Tondeur & Johan van Braak, In Journal of Computer Assisted Learning
Technological Pedagogical Content Knowledge (TPACK) has been introduced as a conceptual framework for the knowledge... more Technological Pedagogical Content Knowledge (TPACK) has been introduced as a conceptual framework for the knowledge base teachers need to effectively teach with technology. The framework stems from the notion that technology integration in a specific educational context benefits from a careful alignment of content, pedagogy and the potential of technology, and that teachers who want to integrate technology in their teaching practice therefore need to be com- petent in all three domains. This study is a systematic literature review about TPACK of 55 peer-reviewed journal articles (and one book chapter), published between 2005 and 2011. The purpose of the review was to investigate the theoretical basis and the practical use of TPACK. Findings showed different understandings of TPACK and of technological knowledge. Impli- cations of these different views impacted the way TPACK was measured. Notions about TPACK in subject domains were hardly found in the studies selected for this review. Teacher knowledge (TPACK) and beliefs about pedagogy and technology are intertwined. Both deter- mine whether a teacher decides to teach with technology. Active involvement in (re)design and enactment of technology-enhanced lessons was found as a promising strategy for the develop- ment of TPACK in (student-)teachers. Future directions for research are discussed.
147 views
Seen by:
Intelligence in youth and all-cause-mortality: systematic review with meta-analysis
International Journal of Epidemiology 2010;1–19
Catherine M Calvin,1 Ian J Deary,1* Candida Fenton,2 Beverly A Roberts,1 Geoff Der,2 Nicola Leckenby1 and G David Batty1,2,3
1 Centre for Cognitive Ageing and Cognitive Epidemiology, Department of Psychology, University of Edinburgh, Edinburgh, UK,
2 Medical Research Council Social & Public Health Sciences Unit, Glasgow, UK and
3 Department of Epidemiology & Public Health, University College London, London, UK
*Corresponding author. Centre for Cognitive Ageing and Cognitive Epidemiology, Department of Psychology, University of Edinburgh, 7 George Square, Edinburgh EH8 9JZ, UK. E-mail: ian.deary@ed.ac.uk
Background A number of prospective cohort studies have examined the association between intelligence in childhood or... more
Background A number of prospective cohort studies have examined the association between intelligence in childhood or youth and life expectancy in adulthood; however, the effect size of this association is yet to be quantified and previous reviews require updating.
Methods The systematic review included an electronic search of EMBASE, MEDLINE and PSYCHINFO databases. This yielded 16 unrelated studies that met inclusion criteria, comprising 22 453 deaths among 1 107 022 participants. Heterogeneity was assessed, and fixed effects models were applied to the aggregate data. Publication bias was evaluated, and sensitivity analyses were conducted.
Results A 1-standard deviation (SD) advantage in cognitive test scores was associated with a 24% (95% confidence interval 23–25) lower risk of death, during a 17- to 69-year follow-up. There was little evidence of publication bias (Egger’s intercept = 0.10, P = 0.81), and the intelligence–mortality association was similar for men and women. Adjustment for childhood socio-economic status (SES) in the nine studies containing these data had almost no impact on this relationship, suggesting that this is not a confounder of the intelligence–mortality association. Controlling for adult SES in five studies and for education in six studies attenuated the intelligence–mortality hazard ratios by 34 and 54%, respectively.
Conclusions Future investigations should address the extent to which attenuation of the intelligence–mortality link by adult SES indicators is due to mediation, over-adjustment and/or confounding. The explanation(s) for association between higher early-life intelligence and lower risk of adult mortality require further elucidation.
Dominance and plumage traits: meta-analysis and meta-regression analysis
Santos ESA, Scheck D & Nakagawa S. 2011. Animal Behaviour, 82, 3-19.
The study of avian signalling systems is an important avenue for research on animal communication. More specifically,... more The study of avian signalling systems is an important avenue for research on animal communication. More specifically, the use of plumage traits to signal status or fighting ability has been extensively investigated for over 30 years. Yet, little is known about how strong and general the relationship between dominance and plumage traits is across different bird species. We conducted meta-analysis and meta-regression analysis to investigate the relationship between dominance and plumage characteristics, using published literature on avian species. Our analyses tested possible moderators of this relationship, such as plumage trait and plumage colour, plumage manipulation, study location, type of dominance interaction, the use of model birds, type of dominance assessment, and dominance context (breeding or non-breeding). Our results revealed a significant effect size for the association between dominance and plumage. Our analyses revealed that the relationship between dominance and plumage was significantly influenced by the dominance assessment method used in the source study (direct or indirect). Furthermore, our analyses provide evidence that the role of plumage signalling may be similarly important during the breeding and non-breeding seasons. We conclude that plumage signalling is a ubiquitous form of communicating status or fighting ability across bird species with different plumage types.
Methodological issues and advances in biological meta-analysis
Nakagawa, S. & Santos, ESA. 2012. Evolutionary Ecology. DOI: 10.1007/s10682-012-9555-5
Meta-analysis has changed the way researchers conduct literature reviews not only in medical and social sciences but... more Meta-analysis has changed the way researchers conduct literature reviews not only in medical and social sciences but also in biological sciences. Meta-analysis in biological sciences, especially in ecology and evolution (which we refer to as ‘biological’ meta-analysis) faces somewhat different methodological problems from its counterparts in medical and social sciences, where meta-analytic techniques were originally developed. The main reason for such differences is that biological meta-analysis often integrates complex data composed of multiple strata with, for example, different measurements and a variety of species. Here, we review methodological issues and advancements in biological meta-analysis, focusing on three topics: (1) non-independence arising from multiple effect sizes obtained in single studies and from phylogenetic relatedness, (2) detecting and accounting for heterogeneity, and (3) identifying publication bias and measuring its impact. We show how the marriage between mixed-effects (hierarchical/multilevel) models and phylogenetic comparative methods has resolved most of the issues under discussion. Furthermore, we introduce the concept of across-study and within-study meta-analysis, and propose how the use of within-study meta-analysis can improve many empirical studies typical of ecology and evolution.
Supporting self-regulated learning in computer-based learning environments: systematic review of effects of scaffolding in the domain of science education
by Jo Tondeur
Authors: Anneline Devolder, Johan van Braak, & Jo Tondeur
In Journal of Computer Assisted Learning
Despite the widespread assumption that students require scaffolding support for self-regulated learning (SRL)... more Despite the widespread assumption that students require scaffolding support for self-regulated learning (SRL) processes in computer-based learning environments (CBLEs), there is little clarity as to which types of scaffolds are most effective. This study offers a literature review covering the various scaffolds that support SRL processes in the domain of science education. Effective scaffolds are categorized and discussed according to the different areas and phases of SRL. The results reveal that most studies on scaffolding processes focus on cognition, whereas few focus on the non-cognitive areas of SRL. In the field of cognition, prompts appear to be the most effective scaffolds, especially for processes during the control phase. This review also shows that studies have paid little attention to scaffold designs, learner characteristics, or various task characteristics, despite the fact that these variables have been found to have a sig- nificant influence. We conclude with the implications of our results on future design and research in the field of SRL using CBLEs.
Routine development of objectively derived search strategies
Hausner E, Waffenschmidt S, Kaiser T and Simon M Systematic Reviews 2012, 1:19 (29 February 2012)
BACKGROUND:Over the past few years, information retrieval has become more and more professionalized and information... more BACKGROUND:Over the past few years, information retrieval has become more and more professionalized and information specialists are considered full members of a research team conducting systematic reviews. Research groups preparing systematic reviews and clinical practice guidelines have been the driving force in the development of search strategies, but open questions remain regarding the transparency of the development process and the available resources. An empirically guided approach to the development of a search strategy provides a way to increase transparency and efficiency.METHODS:The aim of this paper is to describe the empirically guided development process for search strategies as applied by the German Institute for Quality and Efficiency in Health Care (Institut fur Qualitat und Wirtschaftlichkeit im Gesundheitswesen, IQWiG). This strategy consists of the following steps: generation of a test set, as well as development, validation, and standardized documentation of the search strategy.RESULTS:We illustrated our approach by means of an example, i.e. a search for literature on brachytherapy in patients with prostate cancer. For this purpose, a test set was generated including a total of 38 references from 3 systematic reviews. The development set for the generation of the strategy included 25 references. After application of text-analytic procedures a strategy was developed including all references of the development set. In order to test the search strategy on an independent set of references, the remaining 13 references from the test set (validation set) were used; the validation set was also completely identified.DISCUSSION:Our conclusion is that an objectively derived approach, similar to that used in search filter development, is a feasible way to develop and validate reliable search strategies. Besides creating high-quality strategies, the widespread application of this approach would result in a substantial increase in the transparency of the development process of search strategies.},
17 views