1 Syllabus

Please see this webpage for more details on the Syllabus.

2 Chapt. 1 – The Reproducibility Crisis

2.1 Goal

In this chapter, we engage in group activities and literature reading to explore the concept of reproducible science, including its challenges, benefits, and practical implementation in research. Overall, this chapter aims to provide a broader understanding of reproducible science and the context that underpins the material covered throughout the course.

The chapter is subdivided into two parts:

  • PART A: What is Reproducible Science?
  • PART B: What Factors Break Reproducibility?

2.2 Learning Outcomes

  • Understand the definitions of reproducibility and replicability in scientific research
  • Identify the importance of reproducible science for research integrity and advancements
  • Recognize the challenges to achieving reproducibility and potential solutions
  • Engage with practical tools and strategies to promote reproducible research practices

2.3 Online Resources

This list of online resources supports our group activities:

  • Reproducibility and Replicability in Science - A report published by the National Academies of Sciences, Engineering, and Medicine.
  • Retraction Watch - Tracking retractions as a window into the scientific process.
  • PubPeer - An online journal club where researchers are posting comments on articles.
  • PubMed - This database comprises more than 39 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites.
  • RESCIENCE C - Reproducible Science is good. Replicated Science is better.
  • Peer Review Week - Peer Review Week is a community-led yearly global virtual event celebrating the essential role that peer review plays in maintaining research quality. The event brings together individuals, institutions, and organizations committed to sharing the central message that quality peer review in whatever shape or form it may take is critical to scholarly communication.
  • Research Integrity and Peer Review - An international, open access, peer reviewed journal that encompasses all aspects of integrity in research publication, including peer review, study reporting, and research and publication ethics.
  • Tools such as ChatGPT threaten transparent science; here are our ground rules for their use - A Nature editorial discussing the inclusion of AI in research articles.

2.4 What is Reproducible Science?

To investigate this question, students engage in self-reflection and group activities designed to explore the meaning of reproducible science and its implementation in research and scientific publications.

Disclaimer: The instructor has provided definitions and other supporting materials (in collapsible boxes) throughout this section. However, students are advised not to read these materials in advance, in order to complete the exercises sincerely and derive the greatest benefit from the experience.

2.4.1 Class Outline

The structure of this class, held over two sessions, is as follows:

  1. Introduction and Definitions (30 minutes)
  2. Case Studies and Challenges (80 minutes over two sessions)
  3. Best Practices and Open Science Tools (20 minutes)
  4. Wrap-up and Future Directions (10 minutes)

The first two sections will be covered during Session 1, while the remaining sections will be addressed in Session 2.
Please ensure that you have identified the reproducibility challenges in your assigned publication (see Case Studies and Challenges) prior to attending the second session.

2.4.2 Introduction and Definitions

2.4.2.1 ✍️ Individual Self-Reflection (5 minutes)

Students reflect on what “reproducible science” means to them and write down a personal definition.

2.4.2.2 🤝 Group Sharing and Discussion (15 minutes)

  • Form small groups (2-3 students)
  • Small groups share and compare their definitions
  • Each group presents insights to the class to foster open discussion

2.4.2.3 📘 Formal Definitions (10 minutes)

Standard definitions are introduced to clarify the distinction between reproducibility and replicability.
Students are invited to relate them to their own research experience.

📌 What is Reproducible Science?

Reproducible science means achieving the same findings using the original data, methods, and code.

Key elements:

  • Transparency – Open access to data, code, and methods.
  • Documentation – Clear steps and research design explanation.
  • Data Availability – Raw data is publicly available.
  • Code Availability – Scripts are shared and well-commented.
  • Standardized Practices – Use of trusted tools and workflows.
🔄 Reproduction vs. Replication

Reproduction = Running the same code on the same data to verify results (like software testing for an entire study).
Replication = Repeating the study with intentional variations (e.g., different species, software, or parameters) to test if conclusions hold.

  • Reproduction confirms that the research is traceable and correctly recorded.
  • Replication tests what aspects of a study matter for the results to hold.
Key insight: Replication is most useful after reproduction is verified.


💭 Think about it: Can you think of examples of scientific studies, methods, or protocols where replication would be particularly important? What would you learn by repeating them with intentional variations?

2.4.3 Case Studies and Challenges

2.4.3.1 👥 Group Work Activity (50 minutes)

Each group is given a real-world publication that illustrates challenges to reproducibility, such as:

  • 🔒 Lack of data/code sharing
    Data and code are not made available, preventing verification and reuse.
  • 🧩 Incomplete methodological detail
    Missing or vague descriptions of protocols, tools, or parameters hinder reproducibility.
  • 📉 Poor statistical practices
    Misuse or misunderstanding of statistical tests can lead to invalid conclusions.
  • 🎯 Publication bias
    Preference for positive or novel results skews the scientific record and limits replication efforts.
  • 🤖 Unclear or unvalidated use of AI tools
    Use of AI (e.g., large language models, predictive algorithms) without documenting models, prompts, parameters, or validation steps reduces transparency and reproducibility.
  • 🧪 Data manipulation or selective reporting
    Deliberate or unintentional exclusion, alteration, or cherry-picking of data that misrepresents findings.

2.4.3.2 📄 Publications Used in This Activity

I assign one of the retracted articles below to each group at the start of the activity. The links are here so that you can find your article again afterwards. The reasons behind each retraction are given at the end of this section — work through your article and reach your own conclusions before opening that table. The articles are also available on our Google Drive.

📝 Task Instructions:

Each group:

  1. Read their assigned publication
  2. Identify specific reproducibility challenges
  3. Investigate the journal where the article was published, including its policies and practices related to research ethics, integrity, transparency, and reproducibility, as well as its impact and influence in the field
  4. Discuss how these challenges and the journal’s practices might impact the broader research community
  5. Brainstorm strategies or solutions to mitigate these issues and improve reproducibility, considering both researchers and journals
💡 Note: Students are encouraged to explore their assigned study online and consult the provided resources.
However, please form your own conclusions before researching externally.

2.4.3.3 📢 Group Presentations & Class Discussion (20 minutes)

  • Each group presents:
    • The reproducibility challenges in their case study
    • Proposed solutions or best practices to address them
  • 🧭 Facilitated Discussion Tip:
    Guide the discussion by drawing links across different case studies — uncover systemic challenges and the interconnected factors that shape reproducibility across disciplines.

2.4.3.4 Reasons Behind the Retractions of Case Studies (10 minutes)

The data supporting the retractions of the case studies are presented in Table 2.1 (sourced from Retraction Watch). Digital Object Identifiers (DOI) and PubMed IDs are provided for each case study.

📊 Show Table 2.1 — the reasons behind each retraction
Table 2.1: Reasons behind the retractions of case studies sourced from the Retraction Watch database.
GroupID Title Journal ArticleType RetractionDOI RetractionPubMedID OriginalPaperDOI OriginalPaperPubMedID Reason
1 Primary Prevention of Cardiovascular Disease with a Mediterranean Diet NEJM: The New England Journal of Medicine Research Article; 10.1056/NEJMc1806491 29897867 10.1056/NEJMoa1200303 23432189 +Error in Analyses;+Error in Methods;+Error in Results and/or Conclusions;+Retract and Replace;
2 Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial International Journal of Antimicrobial Agents Clinical Study; 10.1016/j.ijantimicag.2024.107416 39730229 10.1016/j.ijantimicag.2020.105949 32205204 +Concerns/Issues About Authorship/Affiliation;+Concerns/Issues About Data;+Concerns/Issues about Results and/or Conclusions;+Concerns/Issues about Article;+Concerns/Issues about Human Subject Welfare;+Date of Article and/or Notice Unknown;+Informed/Patient Consent - None/Withdrawn;+Investigation by Journal/Publisher;+Investigation by Third Party;
3 Correlation of Carotid Artery Intima-Media Thickness with Calcium and Phosphorus Metabolism, Parathyroid Hormone, Microinflammatory State, and Cardiovascular Disease BioMed Research International Research Article; 10.1155/2024/9893064 38550095 10.1155/2022/2786147 35313627 +Computer-Aided Content or Computer-Generated Content;+Concerns/Issues About Data;+Concerns/Issues about Referencing/Attributions;+Concerns/Issues about Results and/or Conclusions;+Concerns/Issues with Peer Review;+Investigation by Journal/Publisher;+Investigation by Third Party;+Paper Mill;+Unreliable Results and/or Conclusions;

2.4.4 Best Practices and Open Science Tools

2.4.4.1 🧠 Whiteboard Brainstorm (20 minutes)

Working in 2–3 groups, each at a whiteboard, you will build a two-column list: on the left the issues that break reproducibility, and on the right the best practice or open science tool that addresses each one.

Start from the retracted articles you have just worked on. The case studies in Case Studies and Challenges and the reasons behind their retractions in Table 2.1 are your raw material: every entry in the Reason column of that table is something that somebody failed to prevent. Begin by asking what would have had to be different for your article never to have been retracted.

The order matters — diagnose before you prescribe.

  1. List the issues first (~8 minutes) What actually went wrong in your group’s article, and what does Table 2.1 say went wrong in the other two? Add anything from the challenge types listed in that section, or from your own research experience. Be specific: “the raw data were never deposited anywhere” is workable; “bad practice” is not.

  2. Then pair each issue with a solution (~8 minutes) Work along your list and, for every issue, name the practice, habit or tool that would have prevented it. Some issues will attract several solutions; some solutions will fix several issues. Both are worth noticing.

  3. Report back (~4 minutes) Each group walks the class through its board. We will then compare the boards to see which issues everyone found, and which were spotted by only one group.

💡 One rule: every practice or tool on the right must answer a specific issue on the left. A tool nobody can attach to a problem is a tool you do not yet have a reason to use — and saying so out loud is a perfectly good contribution.

2.4.5 Wrap-up and Future Directions

2.4.5.1 🚀 Key Strategies & Tools (10 minutes)

With your boards from the brainstorm still up, we now compare them against the short list below and turn the discussion towards your own work.

Three questions guide this last part:

  1. What did we all find? Which practices appeared on every board, and which on only one? A tool that only one group thought of is usually the one worth talking about.
  2. What is missing from the list below — and what is missing from your boards? The list is deliberately short. If your board names something it does not, that is not an error on your part; it is a gap in the list, and I would like to hear it.
  3. What will you actually deploy? For each strategy, ask whether it belongs in your research: could you start using it this semester, does it need a tool you do not yet have, or is it not relevant to the kind of data you work with? Naming one practice you will adopt is worth more than agreeing that all of them are good.

Every item below is developed further in Chapt. 5 – Data Management and Reproducible Code, so this is a map of where we are going as much as a summary of where we have been.

Please don’t look at the content of the box before we get here!

✅ Strategies and tools for reproducible science
  • Detailed Protocols: Importance of clear and precise documentation of all steps leading to data production.
  • Lab Notebooks: For improved documentation and data management.
  • Version Control: Using tools like Git for code and data management.
  • Data and Code Repositories: Platforms like Dryad, Zenodo, and Figshare for sharing research materials.

2.5 What Factors Break Reproducibility?

2.5.1 Goal

In Part A, we investigated what reproducible science is, and why it fails. In Part B, we ask which factors break it, and then we do something about them.

We work from two sources of evidence: the survey of 1,576 researchers published by Baker (2016), which reports what the scientific community itself blames for irreproducible work, and the report of the National Academies of Sciences, Engineering, and Medicine (National Academies of Sciences and Medicine, 2019), which sets out what should be done about it.

The goal of Part B is to turn that evidence into a defence. By the end of these two sessions your group will have produced one deliverable: a reproducibility charter — a set of standards your research team commits to, each one chosen to block a specific factor known to break reproducible science. It is a short shared document, and if you want to see what a standard actually looks like before we begin, there are worked examples of charter clauses further down. The diagnosis is where we start; the charter is what we are here to produce.

Your charter opens by saying what it is for. Before any of the standards, it states in three lines what your group means by reproducible science, and what distinguishes reproducibility from replicability (the definitions you worked on in Part A). A charter that sets standards without saying what they are for is just a list of rules; those three lines are what the rules are in service of, which is why they sit at the top. Everything below them has to earn its place by serving them.

2.5.2 Class Outline

This class is run as a single group activity (writing your group’s reproducibility charter) held over two sessions and divided as follows:

  1. Overview of the Group Activity (10 minutes)
  2. What Do You Remember? (10 minutes)
  3. Review the Resources (10 minutes)
  4. The Factors, Together (10 minutes)
  5. Draft Your Charter (30 minutes)
  6. Stress-Test Another Team’s Charter (20 minutes)
  7. Revise and Commit (15 minutes)
  8. Compare the Two Charters (10 minutes)
  9. Wrap-Up: What Did This Take? (5 minutes)

The first five sections will be covered during Session 1, while the remaining four will be addressed in Session 2.
Each group finishes its draft charter as homework between the two sessions, and adds a short block of definitions at the top, because Session 2 opens with another group taking it apart (see Draft Your Charter).

Every one of these sections draws on the Resources set out below: the two publications, the teaching material, and the recommendations of the National Academies. They are the evidence this class is built on — you will check your own answers against them in Review the Resources, and draw on them again for every clause you write into your charter.

2.5.3 Investigated Questions

We will be investigating the following questions:

  • Is there a reproducibility crisis?
  • How much published work in your field is reproducible?
  • What is the scale of the reproducibility crisis?
  • What corrective measures can be implemented?
  • What factors contribute to irreproducible research?
  • What can be done to mitigate the reproducible crisis?

2.5.4 Resources

2.5.4.1 Publications

The PDFs of Baker (2016) and National Academies of Sciences and Medicine (2019) are available in our shared Google Drive at the following paths:

Reproducible_Science > Publications > Baker_Nature_2016.pdf
Reproducible_Science > Publications > Reproducibility_and_Replicability_in_Science_2019.pdf

2.5.4.2 Teaching Material

The presentation associated with this class is below. We work through it in The Factors, Together, after you have committed to your own answers — and it is here afterwards for revision.

2.5.4.3 Recommendations

The most relevant recommendations proposed by the National Academies of Sciences and Medicine (2019) for our course are listed in the table below. It is searchable, so in Review the Resources you can look up the three factors on your shortlist rather than reading the whole table.

2.5.5 👥 Group Activity: Write Your Lab’s Reproducibility Charter

In this session we cover the material of Part B (the survey evidence of Baker (2016), the recommendations of National Academies of Sciences and Medicine (2019), and the questions listed above) and then put it to work. You will use it to write something of your own: a reproducibility charter.

📜 Charter (noun) — a short written document in which a group sets out the standards it binds itself to.

A charter is not advice, and not a summary of good practice. Guidelines describe what researchers in general ought to do; a charter states what this group will always do. Three things follow from that: it is written by the people who will be held to it, each commitment is specific enough that somebody else can tell whether it was kept, and it is public within the group, so colleagues can point to it when it is not.

Research groups, laboratories and consortia write these for exactly this reason — so that standards survive a deadline, a new student, or a change of principal investigator, instead of living in one person’s head.

2.5.5.1 Why we are doing this

You already have the evidence you need. Three things feed this charter, and every clause you write should be traceable to one of them:

  1. What you saw in Part A. You examined retracted articles and the reasons behind their retractions (Table 2.1). Those are documented failures — your charter should make them harder to repeat.
  2. What the field reports. Baker (2016) tells you which factors researchers themselves blame most often. A charter that ignores the commonest causes is not serious.
  3. What has been recommended. National Academies of Sciences and Medicine (2019) sets out what the National Academies concluded should change. You can adopt, adapt or argue with it, but you should know what it says.

The test for every clause is whether it supports the scientific process — whether it makes it more likely that a claim you publish can be checked, repeated and built on by somebody else. That is what reproducibility is for. A standard that is merely tidy, or that only makes your own life easier, is not yet a reproducibility standard.

And this is not a one-off exercise. Nearly everything you write today (data management, documentation, reproducible code, version control, deposition, citation) is developed properly later in this course. Your charter is therefore a map of where we are going: at the end of the semester you should be able to return to it and find that you now know how to implement what you committed to today, and to judge whether you were ambitious enough.

Work in two groups of four. Each group is a research team, and every member owns a section of the charter:

Role Owns the standards for
Data lead How data are generated and recorded: protocols, field and laboratory notes, file naming, storage, backup and metadata
Analysis lead Code and statistics: the analysis plan, what is decided before the data are seen, and how an analysis is made re-runnable
Reporting lead What leaves the group: what is deposited and when, how results and limitations are reported, and how data, software and sources are cited

Between them these three cover the whole data life cycle, so nothing in the charter is unowned. With four of you, one section has two co-leads: agree which, and both of you answer for it. Chapter 3 adds a fourth role, so by then everyone has a section of their own. You are responsible for your section and you will be the one defending it, so choose the role closest to the research you actually do — or the one furthest from it, if you would rather learn something new.

2.5.5.2 🧠 What Do You Remember? (10 minutes)

Books closed. No paper, no slides, no notes, no laptops. You read Baker (2016) for today, and this is where you find out how much of it stayed with you.

Working as a group, agree on four answers and write them on a single sheet:

  1. The scale. Of the 1,576 researchers surveyed, what share reported a significant reproducibility crisis? Give a percentage.
  2. Your own field. What proportion of published work in your field do you believe is reproducible? This one is not recall — it is your own judgement, and there is no answer at the back of the book.
  3. The causes. Which three factors did respondents most often blame for irreproducible research? Name them as precisely as you can.
  4. Your shortlist. From those, a provisional list of the three factors you think are most dangerous to your research. Provisional because you are about to check your answers, and may want to change your mind.

Commit to specifics. “Quite a lot of people” is not an answer; “about 70%” is. Recalling something and getting it wrong fixes the right answer in memory far better than reading it a second time does — which is the whole point of doing this before you open anything.

📄 What you take out of this step: one sheet per group, with four answers on it. You will mark it up in Review the Resources, and your shortlist becomes what the charter has to defend against.

2.5.5.3 📖 Review the Resources (10 minutes)

Now confirm your answers against the evidence. You have committed to four answers from memory; this step is where you find out which of them hold up. The PDFs are on our shared Google Drive as well as linked below.

Work individually, and in silence. Go to Resources and give each of the three a few minutes:

  • the publications — check the figures you have just tried to recall, and read around the ones you got wrong;
  • the teaching material — skim the slides for anything you have not already met;
  • the recommendations — the table is searchable, so look up the three factors on your group’s shortlist.

Mark up your sheet as you go: correct the numbers you got wrong, and note which of your three factors you now want to keep, and which you want to swap.

Read with your own role in mind. You will shortly be drafting as the data, analysis or reporting lead, and this is the only part of the session where you are reading rather than arguing — so note down what you will want when it is your turn to write.

By the end of this step your group should have a corrected sheet and a settled shortlist — the evidence checked, the three factors chosen. That is everything the charter needs to be built on.

2.5.5.4 📊 The Factors, Together (10 minutes)

I will take us through the factors that break reproducibility, drawing on the slides and on the recommendations of National Academies of Sciences and Medicine (2019). Since you have read Baker (2016) already, this is a short synthesis rather than a walk through the paper.

Keep the three factors from the previous step in front of you: if something named here is not on your list, add it — your charter is about to have to deal with it.

2.5.5.5 ✍️ Draft Your Charter (30 minutes)

Write it in a shared document. Before you start, one member of each group creates a Google Doc (preferred, because all three of you can type in it at once) or a Word document, names it Charter_Group1 or Charter_Group2, and saves it in our shared Google Drive at Reproducible_Science > Chapters > Chapter_1 > Part_B. Give it four headings: (i) Definitions, (ii) Data, (iii) Analysis, and (iv) Reporting. Each lead writes under their own section today; the Definitions block is completed as homework (see the end of this step).

Keeping it in that folder is what makes the rest of the activity work: it is how the three of you draft together, how you carry on after class, and how the other group opens your charter to stress-test it.

Each lead drafts the clauses for their own section. Between you, the charter must address all three factors from the first step, and you may consult the recommendations table for wording and for what others have proposed.

Be ready to say where each clause comes from: a failure you saw in Part A, a factor reported by Baker (2016), or a recommendation from National Academies of Sciences and Medicine (2019). A clause nobody can source is usually a habit, not a standard.

The standard is that a clause must be checkable. Somebody outside your group should be able to tell whether you followed it:

  • ❌ “We will document our data well.”
  • ✅ “Every dataset has a README recording who collected it, when, with what instrument, and what each column means, written before the data leave the field laptop.”

For each clause, be ready to say what it costs in time, in effort, or in freedom to change your mind later. A standard nobody can afford is a standard nobody will keep.

You are not starting from nothing. The box below gives each lead a set of questions to work through, one worked example to show the level of detail expected, and where to look in the course material. Aim for two or three clauses per section — six to nine in total is a charter; twenty is a wish-list. The three definitions go at the top, and are written as homework once the clauses are settled.

🧭 Starting points for each section

📜 Definitions — completed as homework, see Introduction and Definitions in Part A. One line each for reproducible science, reproducibility and replicability, in your own words. Write them after your clauses, not before: it is far easier to say what your standards are for once you can see what they actually are.

Everyone: the four strategies in Wrap-up and Future Directions — detailed protocols, laboratory notebooks, version control, and data and code repositories — are the obvious backbone. Your charter should say something about each of them, in the section where it belongs.

📊 Data lead — see Data Management (Chapt. 5) and the data life cycle: create, process, document, preserve, share, reuse.

  • Where do raw data live the moment they are created, and where is the second copy?
  • How would somebody who was not there know what each file, and each column, contains?
  • What has to be written down at the time because it cannot be reconstructed later?
  • What happens to the raw file once cleaning starts?

Example clause: “Raw data are never edited. All cleaning happens in a script, and the raw file is kept read-only.”

💻 Analysis lead — see Reproducible Code (Chapt. 5).

  • Which decisions in an analysis get made after seeing the data, and which should be fixed before?
  • If your laptop disappeared tomorrow, what would somebody need in order to re-run your analysis?
  • How do you record which version of which software produced a given result?
  • How does a number in your manuscript get traced back to the code that produced it?

Example clause: “Every figure and table in a manuscript is produced by code that runs from the raw data with a single command.”

📢 Reporting lead — see Open Science and CARE Principles and Getting Published.

  • What must a reader have in order to check your central claim for themselves?
  • When does that become available — at submission, at acceptance, on request, or never?
  • How are data, code and software credited, and who is an author?
  • What do you report when an analysis did not work, or was abandoned?

Example clause: “Data and code are deposited before submission, not after acceptance, and every software package is cited with its version.”

📌 Before Session 2: finish the draft. You are not expected to leave today with a complete charter — you are expected to leave with every section started and the hard arguments had. Each lead completes their own clauses as homework in the shared document, so the group arrives with one agreed draft rather than three sets of notes. Have it finished before the next session starts — the other group opens it straight away.

Then add the Definitions block at the top of the charter. Three entries, one line each:
  • Reproducible science — what it is, in a single sentence;
  • Reproducibility — what it means to reproduce a study;
  • Replicability — and how that differs.
Use the definitions you worked on in class in Introduction and Definitions (Part A), written in your own words rather than copied. As set out in the Goal, these lines are what the rest of your charter is in service of — and in the next session the other group will read them first, then check whether your clauses actually deliver them.

2.5.5.6 🔍 Stress-Test Another Team’s Charter (20 minutes)

Open the other group’s charter from Reproducible_Science > Chapters > Chapter_1 > Part_B — you review theirs, they review yours.

Each of you reviews the section you wrote yourselves: the data lead reads their Data section, the analysis lead their Analysis section, the reporting lead their Reporting section. You have just written standards for that part of the research process, so you are the best placed to see what is missing from theirs — and you will be defending your own version of it in a few minutes.

Leave your challenges as comments in their document. Select the text of the clause you are questioning, then Insert > Comment (Cmd/Ctrl + Alt + M), so that each objection is anchored to the clause it concerns rather than collected in one note at the end. Aim for two or three substantive comments on your section, each saying why: “this would slip the moment fieldwork overran” is useful; “unclear” is not.

Work on your own section for the first 15 minutes (individually), then spend the last 5 minutes together (as a group) agreeing an overall evaluation of the charter. Your job now is to be the sceptic:

  1. (on your own section) Which clause breaks first when a deadline is three days away?
  2. Which clause is unenforceable — how would anyone know it had been broken?
  3. What is missing? You wrote standards for the same section, so start there: what did you cover that they did not? Then look at the retracted articles and Table 2.1 in Part A — would this charter have prevented those retractions?
  4. (together, in the last 5 minutes) Does the charter as a whole serve its definitions? Read their Definitions block at the top, then the standards below it. Does this charter actually deliver what the group says reproducible science is? Leave that verdict as a single comment on the document, agreed between the three of you.

Then hand the charter back. You answer the challenges on your own section in the next step.

Resolve nothing and delete nothing. The threads are a record of which standards survived scrutiny and which did not, and we will read them again when we compare the two charters.

2.5.5.7 ✅ Revise and Commit (15 minutes)

Before you change a single word, put the document into tracked mode. In Google Docs, click the pencil icon at the top right and choose Suggesting; in Word, Review > Track Changes. Everything you do from here is then recorded as a suggestion rather than silently overwriting what you wrote before the review — so you can see exactly what the stress-test changed, and so can I.

Then work in this order:

  1. Reply to every comment on your own section, first. Answer them one at a time, in the thread where each was raised: defend the clause, or concede it. Both are good answers; neither is a loss, and conceding to a good objection is the harder skill. Leave nothing unanswered — an objection nobody replied to is one the group has quietly ignored.
  2. Then revise the text. Working in suggestion mode, rewrite the clauses you conceded and tighten any that turned out to be unenforceable. Because your replies are already in the threads, each tracked change sits next to the argument that produced it.

Still resolve nothing and delete nothing. The comments, your replies and the tracked changes together are the record of how your charter reached its final state — which is exactly what a reproducible revision history looks like, and the same habit you will apply to code and manuscripts later in this course.

Once your charter has been revised, each student states, out loud, one clause from their own section that they will apply to their own project this semester — starting now, not in principle.

💡 Why a charter? Diagnosing other people’s failures is comparatively easy. Deciding what your group will always do — knowing it will cost you something, and that colleagues can hold you to it — is what a research lead actually does. That is the skill this activity is for.

2.5.5.8 🤝 Compare the Two Charters (10 minutes)

Both charters now go up side by side. We are not going to merge them into one class charter — we are going to work out what it would take to get there, and what stands in the way.

First, in your own group (~4 minutes) — read the other charter through once more and agree two answers between the three of you:

  • What did we hold in common? Which standards appear, in some form, in both documents?
  • Where did we diverge? Which clause did we write that they did not, or write very differently?

Then, as a class (~6 minutes) — each group puts those two answers to the other, and we discuss:

  1. Convergence. Two teams working independently from the same evidence and landing on the same standard is the strongest signal we have that it matters — that is consensus, not coincidence. Which standards did that happen to?
  2. Divergence. For each clause only one group wrote: is it a blind spot in the other charter, or a real difference between the kinds of research you do? Field data, code and specimens do not face identical risks, and a standard that is essential in one laboratory can be irrelevant in another.
  3. Could these two documents become one? Where would you have to compromise, which clause would you refuse to give up, and what would a charter have to leave out in order for a whole department to sign it? You do not have to resolve this — but notice how much harder agreement becomes as the group grows.
📌 Your charter stays yours. We are not producing a single class document, because the point is not the artefact — it is that you can now argue about standards with evidence, and see why a field, a journal or a funder finds it so difficult to agree on them. What you have written is developed properly in Chapt. 4 — Open Science and CARE Principles, in Data Management and Reproducible Code in Chapt. 5, and in Chapt. 6 — Getting Published and Peer Review. We will come back to your charter as we go: each time, you should be able to take a clause and say how you would now actually implement it — and, by December, whether you were ambitious enough.

2.5.5.9 🪞 Wrap-Up: What Did This Take? (5 minutes)

To close Part B, each group answers two questions out loud — briefly, and honestly:

  1. What did you learn from writing a charter that you would not have learned from reading about reproducibility?
  2. What was hard? Which part of this actually gave you trouble — agreeing on wording, making a standard checkable, having your clause challenged, deciding what you were willing to commit to?

There are no right answers here, and “we could not agree” is a finding, not a failure. Naming the difficulty is worth as much as the document you produced: the obstacles you hit in ninety minutes are the same ones that keep laboratories, journals and whole fields from adopting the practices everyone agrees are correct.

3 Chapt. 2 – R, R Markdown and User-Defined Functions

3.1 Introduction

Chapter 1 established why reproducibility matters and what breaks it. This chapter is where we begin building the tools to do something about it.

Reproducibility is not all-or-nothing: it is a spectrum (Figure 3.1). A publication on its own sits at the not reproducible end. Releasing the data moves you along it; releasing data and code moves you further. Linked, executable data and code (a document in which the analysis actually runs as the text is written) approaches the gold standard.

This chapter teaches you to work at that end of the spectrum, using R Markdown in RStudio (RStudio Team, 2020): a single environment where text, data, code, figures, tables and references live in one document that anyone can re-run and get your results.

It is also where the reproducibility charter you wrote in Chapt. 1, Part B stops being a statement of intent. Most of the standards you committed to (that raw data are never edited, that every figure is produced by code, that an analysis can be re-run from scratch) are difficult to keep through discipline alone, because nothing stops you breaking them. Written into an R Markdown document, they become structural: the document simply will not knit if the code does not run, and a figure cannot drift from the analysis that made it when the analysis is sitting directly above it. This chapter is how your charter gets enforced.

A left-to-right gradient scale from blue to gold: a blue box labeled Publication only sits at the not reproducible end, a wide middle box labeled Publication plus contains three graded levels (Data, then Data and code, then Linked and executable data and code), and a gold box labeled Full replication sits at the gold standard end.

Figure 3.1: The spectrum of reproducibility.

3.1.1 Aim

This chapter aims to equip students with the essential skills required to effectively use R Markdown for integrating text, code, figures, tables, and references into a cohesive, reproducible document. The final output can be rendered into multiple formats—including PDF, HTML, or Word—to streamline the communication and sharing of research findings.

This tutorial provides students with the foundational knowledge necessary to complete their bioinformatics tutorials (PART 2) and individual projects (PART 3).

3.1.2 Objectives

The chapter is divided into six parts, each with specific learning objectives:

  • PART A: Learning the Basics
  • PART B: Setting Your R Markdown Document
  • PART C: Tables, Figures and References
  • PART D: Advanced R and R Markdown Settings
  • PART E: User-Defined Functions in R
  • PART F: Interactive Tutorials

3.1.3 Resources and Software

Everything you need in order to follow this chapter is collected here: the files, the software to install, and where to read further.

3.1.3.1 Supporting Files

Files supporting this chapter are available on Google Drive.

3.1.3.2 Install R Markdown Software

Software and packages required to perform this tutorial are detailed below. Students should install those software and packages on their personal computers to be able to complete this course. Additional packages might need to be installed and the instructor will provide guidance on how to install those as part of the forthcoming tutorials.

  • R: https://www.r-project.org
  • R packages: bookdown, knitr and R Markdown. Use the following R command to install those packages:
install.packages(c("bookdown", "knitr", "rmarkdown"))
  • RStudio: https://www.rstudio.com/products/rstudio/download/
  • TeX: This software is required to compile document into pdf format. Please install MiKTeX on Windows, MacTeX on OS X and TeXLive on Linux. For this class, you are not requested to install this software, which takes significant hard drive space and is harder to operate on Windows OS.

NOTE: The instructor is using the following version of RStudio: Version 2025.05.0+496 (2025.05.0+496). If your computer is experiencing issues running the latest version of the software, you can install previous versions here.

3.1.3.3 Web Resources

Below are URLs to web resources that provide key information related to Chapter 2:

3.2 PART A: Learning the Basics

In this part, we will provide a survey of the procedures to create and render (or knitting) your first R Markdown document.

3.2.1 Learning Outcomes

This tutorial provides students with the opportunity to learn how to:

3.2.2 What is RStudio?

RStudio (RStudio Team, 2020) is an integrated development environment (IDE) that allows you to interact with R more efficiently. While similar to the standard R GUI, RStudio is significantly more user-friendly. It offers more drop-down menus, tabbed windows, and extensive customization options (see Figure 3.2).

Detailed information on using RStudio can be found on RStudio’s website.

Screenshot of the RStudio window with labels overlaid on each of its four panes: Source with an open .Rmd file top left, Console/Terminal/RMarkdown bottom left, Environment/History/Connections top right, and Files/Plots/Packages/Help bottom right.

Figure 3.2: Snapshot of the RStudio environment showing the four windows and their content.

3.2.3 Introduction to R Markdown

Markdown is a simple formatting syntax used for authoring HTML, PDF, and Microsoft Word documents. It is implemented in the rmarkdown package. An R Markdown document is typically divided into three sections (see Figure 3.3):

  1. YAML metadata section: This section provides high-level information about the output format of the R Markdown file. The information defined here is used by the Pandoc program to format the final document (see Figure 3.3).
  2. Main body text: This section represents the core content of your document or publication and uses Markdown syntax.
  3. Code chunks: This section allows you to import and analyze data, and to produce figures and tables that are directly embedded in the output document.
Screenshot of an R Markdown file open in the RStudio source editor, with overlaid labels marking the YAML metadata section between the triple dashes at the top, the publication core text in the middle, a shaded code chunk labeled importing data, and a second shaded chunk labeled analyze data and output figures and tables.

Figure 3.3: Example of an R Markdown file showing the three major sections.

3.2.3.1 Structuring a Report Around the Scientific Process

Those three parts are not just a file format. They let one document carry both the narrative of a study and the analysis that produced it. This is what moves a report towards the linked and executable data and code end of the reproducibility spectrum (Figure 3.1): the text explains what was done and why, and the code chunks actually do it, in the same file, in the order a reader follows.

Two things follow, and you should design your documents around both.

The Markdown text can mirror the structure of a scientific article. Use headers to lay out the familiar sections (Introduction, Materials and Methods, Results, Discussion, References) so that your report reads as a paper rather than as a script with comments. You already know this structure from the articles you examined in Chapt. 1; here you are writing into it.

The code chunks can follow the steps of the scientific process, placed under the section each belongs to (Table 3.1). Those steps are the same ones you will work through in PART 2 of this course (Figure 3.4): importing and tidying the data, then exploring it by transforming, visualizing and modelling, and finally communicating the result.

Table 3.1: How the sections of a scientific report map onto the code that produces them.
Section of the report What the code chunks there do
Materials and Methods Import the raw data, tidy and transform it, document how it was collected
Results Run the analyses, produce the tables and figures, report the statistics
Discussion Draw on those same results, referenced rather than retyped
References Cite the packages and sources used, generated from a bibliography file
Diagram of a reproducible data analysis workflow. On the left, a box labelled Prepare contains Import followed by Tidy. An arrow leads to a larger box labelled Explore and understand, in which Transform, Visualize and Model are connected in a repeating cycle. An arrow then leads out of that box to Communicate. The entire workflow is enclosed in an outer box labelled Program.

Figure 3.4: Overview of the workflow studied in PART 2, and the steps your code chunks implement.

Written this way, the document is the workflow: the Methods section does not merely describe how the data were tidied, it contains the code that tidied them, and a figure in the Results is produced by the chunk sitting directly above it rather than pasted in from elsewhere. Nothing can drift out of step with the analysis, because there is only one copy of it.

This is also how a charter clause becomes something you cannot quietly break. Look back at what your group committed to in Part B and notice how much of it this structure enforces on your behalf:

  • “Every figure and table in a manuscript is produced by code that runs from the raw data”. The chunk sits under the Results heading and produces the figure when the document knits. There is no other way for it to get there.
  • “Raw data are never edited; all cleaning happens in a script”. The cleaning code lives in Materials and Methods, so the edit history is the script itself.
  • “Every software package used is cited with its version”, generated from your bibliography file rather than remembered at submission.

A standard that depends on somebody remembering it will eventually be forgotten. A standard the document refuses to knit without is kept. That is the difference between writing a charter and working to one, and it is why this chapter comes straight after Part B.

This is the structure you will build across the rest of this chapter, and the one your bioinformatics tutorial and individual project are expected to follow.

3.2.4 Your First R Markdown Document

In this section, you will learn how to create and knit your first R Markdown document.

3.2.4.1 Create Document

To create an R Markdown document, follow these steps in RStudio:

  1. Navigate to: File -> New File -> R Markdown...
  2. Enter a title for your document and set the output format to HTML (see Figure 3.5)
    • Title: Chapter 2 - Part A (This is the title of your document, not the name of your file; see below)
    • Note: We will knit documents to the HTML format, which is the default output in RStudio. If you prefer to generate a PDF document, you must have a version of the TeX program installed on your computer (see Figure 3.5).
  3. Save the .Rmd file by selecting: File -> Save As...
    • File name: Chapter_2_part_A.Rmd
    • Project path: Reproducible_Science/Chapters/Chapter_2
    • Warning: Because the knitting process generates multiple files, you should save your .Rmd file in a dedicated project folder.
Screenshot of the RStudio New R Markdown dialog box, with Document selected in the left-hand list, a Title field, a date field, HTML selected among the Default Output Format radio buttons, and Create Empty Document, OK and Cancel buttons along the bottom.

Figure 3.5: Snapshot of window to create an R Markdown file.

3.2.4.2 Knit Document

To knit or render your R Markdown document into an HTML file, follow these steps in RStudio:

  1. Click the Knit button (Figure 3.3) in the top toolbar to render the document.
  2. There are several output format options; however, if you simply click the button, RStudio will automatically knit the document using the settings specified in the YAML metadata section (Figure 3.3).
  3. The output file will be created in the same directory as your .Rmd file (Figure 3.6). You can monitor the progress in the R Markdown console.

💡 Info If the knitting process fails, error messages will appear in the Render console, often indicating the line in the script where the error occurred (although this is not always the case). These messages are helpful for debugging your R Markdown document.

Screenshot of a macOS Finder list view showing an expanded folder named Chapter_2 containing two files, Chapter_2_part_A.html and Chapter_2_part_A.Rmd.

Figure 3.6: Snapshot of your project folder with the knitted html document.

3.2.4.3 How Does the Knitting Process Work?

When you knit your document, R Markdown passes the .Rmd file to the R knitr package, which executes all code chunks and generates a new Markdown (.md) file. This Markdown file includes both the code and its output (see Figure 3.7).

The .md file created by knitr is then processed by the Pandoc program, which converts it into the final output format (e.g., HTML, PDF, or Word) as illustrated in Figure 3.7.

A left-to-right pipeline of five stages joined by arrows: an .Rmd file icon, a knitr box, an .md file icon, a pandoc box, and finally a stack of output document icons.

Figure 3.7: R Markdown flow.

3.2.5 How to Work Through the Rest of PART A

In this section, we will focus on learning the syntax and protocols needed to create headers, comments, lists, formatted text, hyperlinks, code chunks, inline code and spell checking.

You already have the document you need. If you completed Your First R Markdown Document, you created and knitted Chapter_2_part_A.Rmd. That is the file you will keep working in: every section below adds something new to it, so by the end of PART A you will have one document containing all of the syntax you have learned. If you have not created that file yet, go back and do it now, because nothing below will work without it.

🛑 Do this first: go to Before Starting and clear out the template content of your .Rmd file. If you skip this step, the default R Markdown text will still be sitting in your document and your own headers, lists and code chunks will be mixed in with it, which makes the exercises confusing to knit and to read.

Then work through the sections on your own, at your own pace. Each one follows the same shape:

  1. Syntax: what to type, shown in a code block.
  2. Output: what that syntax produces once rendered.
  3. 💡 Your turn: an exercise to add to your own document, ending with a knit and something specific to check.
  4. ✅ Solution: a worked example of the syntax that answers the exercise, hidden behind a toggle. Open it once you have tried the exercise yourself, or if you get stuck.

Do the exercise before moving to the next section rather than reading ahead, and knit every time. Knitting often means that when something breaks you know it was the last thing you typed, which is far easier to fix than a document with twenty untested changes in it.

💡 Ask for help. I am in the room for exactly this. If your document will not knit, or the output does not look the way you expected, raise your hand rather than working around it. A document that refuses to knit is completely normal and the error message in the R Markdown console usually names the line that caused it. Getting used to reading those messages is part of what you are learning here.

We have 60 minutes of class time for this. Each section below carries a time in grey, in the same way as the group activities in Chapt. 1. Those timings add up to the hour and are there so that you (and I) can tell whether we are keeping pace:

Section Time
Before Starting 3 min
1 Headers 6 min
2 Inside text commenting 5 min
3 Lists 10 min
4 Italicized, bold, and combined formatting 6 min
5 Creating hyperlinks 6 min
6 Embedded code chunks 14 min
7 Inline code 6 min
8 Spell checking 4 min
Total 60 min

They are ordered deliberately: later exercises ask you to add content underneath headers you created earlier, so working out of order leaves you with nowhere to put things.

Treat the times as a pace, not a deadline. If a section takes you longer, that is worth knowing and worth telling me, because it usually means something is genuinely unclear rather than that you are slow. If you finish early, open the ✅ Solution and compare it with what you wrote, or help the person next to you. I will call out roughly where we should be as we go.

Additional syntax and formatting options are available in the R Markdown Reference Guide, which you can access in RStudio as follows:

  • Select: Help -> Cheatsheets -> R Markdown Reference Guide

Note: The Cheatsheets section also provides access to other helpful resources related to R Markdown and data manipulation. These documents will be very useful throughout this course.

3.2.6 Before Starting (3 minutes)

This is the step to complete before any of the syntax exercises. Open the Chapter_2_part_A.Rmd file you created earlier and delete its content from line 11 (## R Markdown) to the end of the document, keeping the YAML header and the setup code chunk above it (see Figure 3.8). You are left with an empty document that is ready to knit, and everything you add from here is your own.

Screenshot of the RStudio source editor showing the top of Chapter_2_part_A.Rmd, with the YAML header giving the title, html_document output and a date, followed by a shaded setup code chunk calling knitr opts_chunk set echo TRUE, then the default R Markdown headings and text.

Figure 3.8: R Markdown of Chapter 2 - Part A.

3.2.7 Headers (6 minutes)

Below is the syntax to create headers at three levels:

Syntax:

The "#" refers to the level of the header
# Header 1

## Header 2  

### Header 3

💡 Info

  • While Markdown headers will render correctly without blank lines, it is best practice to include blank lines both before and after headers. Doing so improves the readability of the raw Markdown and helps minimize knitting quirks. Overall, including a blank line before a header (except at the very start of a document) and after it is not strictly required, but it makes your Markdown cleaner and more consistent.

💡 Your turn to practice this syntax in your document by:

  • Including 2 level 1 headers
  • Embedding 2 level 2 headers under each level 1 header
  • Knit your document and check that each level 1 header is larger than the level 2 headers nested under it

I let you pick great names for your headers!

✅ Solution 
# Reproducible science

## What it means

## Why it matters

# R Markdown

## The three sections

## Knitting a document

3.2.8 Inside Text Commenting (5 minutes)

Markdown does not have a built-in syntax for adding comments within text, but you can use HTML-style comments instead.

Syntax:

# HTML syntax to comment inside text
<!-- COMMENT -->

This syntax is typically used to highlight areas of the text that need revision, clarification, or further work. Comments inserted this way will not be visible in the knitted document.

You can learn more about this HTML syntax on this webpage.

💡 Your turn to practice this syntax in your document by:

  • Adding comments describing the content included under each level 1 header
  • Knit your document and check that your comments are not visible in the output

✅ Solution 
# Reproducible science

<!-- This section covers definitions and why reproducibility fails. -->

# R Markdown

<!-- TO DO: add an example of a YAML header here. -->

3.2.9 Lists (10 minutes)

There are two types of lists:

  • Unordered
  • Ordered

3.2.9.1 Syntax for unordered lists (~5 minutes)

Syntax:

* unordered list
* item 2
  + sub-item 1
  + sub-item 2

Note: For each sub-level include two tabs to create the hierarchy.

Output:

  • unordered list
  • item 2
    • sub-item 1
    • sub-item 2

💡 Your turn to practice this syntax in your document by:

  • Including a 2-level unordered list under the first level 1 header
  • Knit your document and check that the sub-items are indented under their parent item

✅ Solution 
* Data
* Code
  + Analysis scripts
  + User-defined functions

3.2.9.2 Syntax for ordered lists (~5 minutes)

Syntax:

1. ordered list
2. item 2
   + sub-item 1
   + sub-item 2

Output:

  1. ordered list
  2. item 2
    • sub-item 1
    • sub-item 2

💡 Your turn to practice this syntax in your document by:

  • Including a 2-level ordered list under the second level 1 header
  • Knit your document and check that the items are numbered and the sub-items are not

✅ Solution 
1. Import the data
2. Tidy the data
   + Rename the columns
   + Drop incomplete records

3.2.10 Italicize and Bold Words (6 minutes)

The following syntax renders text in italics, bold, or both italics and bold.

Syntax:

#Syntax for italics
*italics*

#Syntax for bold
**bold**

#Syntax for italic and bold
***italic and bold***

Output:

  • Italic
  • Bold
  • Italic and bold

💡 Your turn to practice this syntax in your document by:

  • Writing a sentence in your R Markdown document that includes an italicized word
  • Writing a sentence that includes a word or phrase in bold
  • Writing a sentence where a word or phrase is both bold and italicized
  • Knit your document and check all three sentences render differently from one another

✅ Solution 
Reproducibility is *not* the same as replicability.

Raw data are **never** edited by hand.

Every figure is ***produced by code*** that runs from the raw data.

3.2.12 Include Code Chunks (14 minutes)

One of the most exciting features of working with the R Markdown format is the ability to directly embed the output of R code into the compiled document (see Figure 3.7).

In other words, when you compile your .Rmd file, R Markdown will automatically execute each code chunk and inline code expression (see example below), and insert their results into the final document.

If the output is a table or a figure, you can assign a label to it (by adding metadata within the code chunk; see Part B) and refer to it later in your PDF or HTML document. This process—known as cross-referencing—is made possible through the \@ref() function, which is implemented in the R bookdown package.

3.2.12.1 Code Chunk (~7 minutes)

Syntax: a code chunk can be inserted into your document by any of the following methods.

  • Using the keyboard shortcut Ctrl + Alt + I (on macOS: Cmd + Option + I).
  • Clicking the Insert button in the editor toolbar.
  • Typing ```{r} to open the chunk and ``` to close it.

By default, the code chunk expects R code, but you can also insert chunks for other programming languages (e.g., Bash, Python).

💡 Your turn to practice this syntax in your document by:

  • Creating a code chunk in your R Markdown document that prints the message “Hello world”
  • Knit your document and check that both the code and its output appear in the result

✅ Solution 
```{r}
print("Hello world")
```

3.2.12.2 Chunk Options (~7 minutes)

Chunk output can be customized with knitr options arguments set in the {} of a code chunk header. In the examples displayed in Figure 3.9 five arguments are used:

  • include = FALSE prevents code and results from appearing in the finished file. R Markdown still runs the code in the chunk, and the results can be used by other chunks.
  • echo = FALSE prevents code, but not the results from appearing in the finished file. This is a useful way to embed figures.
  • message = FALSE prevents messages that are generated by code from appearing in the finished file.
  • warning = FALSE prevents warnings that are generated by code from appearing in the finished.
  • fig.cap = "..." adds a caption to graphical results.

We will delve more into chunk options in part D of chapter 2, but in the meantime please see the R Markdown Reference Guide for more details.

Screenshot of RStudio split between the source editor on the left, showing an .Rmd file whose code chunks carry different options such as include equals FALSE, message and warning equals FALSE, and fig.cap, and the Viewer pane on the right showing the knitted page with a purple-to-yellow heatmap and its caption.

Figure 3.9: Example of code chunks.

💡 Your turn to practice this syntax in your document by:

  • Editing the code chunk you created earlier (that prints “Hello world”) so that the code is hidden when you knit the document, but the output is still displayed
  • Knit your document and compare it with the previous version: the chunk should now show “Hello world” with no code above it

✅ Solution 
```{r echo=FALSE}
print("Hello world")
```

3.2.13 Inline Code (6 minutes)

Code results can also be inserted directly into the text of a .Rmd file by enclosing the code as follows.

Syntax:

# Syntax for inline code
``r 2+2`` # Please amend to include only 1 back tick

Output: 4

R Markdown will always:

  • Display the results of inline code, but not the code.
  • Apply relevant text formatting to the results.

As a result, inline output is indistinguishable from the surrounding text.

Warning: Inline expressions do not take knitr options and is therefore less versatile. We usually use inline code to perform simple stats (e.g. 4x4; 16)

💡 Your turn to practice this syntax in your document by:

  • Editing a sentence in your R Markdown text to include the output of the line of code 2 + 2
  • Knit your document and check that the sentence reads naturally, with the number 4 in place and no code visible

✅ Solution 
Adding two and two gives `r 2+2`, which R Markdown computes when the document is knitted.

3.2.14 Check Spelling (4 minutes)

There are three ways to access spell checking in an R Markdown document in RStudio:

Syntax: there is none to learn here. Spell checking is a tool in RStudio rather than Markdown syntax, and it can be reached in three ways:

  1. A spell check button to the right of the save button.
  2. Edit > Check Spelling...
  3. The F7 key.

💡 Your turn to practice this in your document by:

  • Running a spell check over everything you have written in this tutorial and correcting what it finds
  • Knit your document one last time and read it through: this is the version your reader sees, and it should now contain headers, comments, lists, formatted text, hyperlinks, a code chunk and inline code

3.3 PART B: Setting Your R Markdown Document

3.3.1 Aim

The aim of this tutorial is to develop an R Markdown document that can be used as a template to produce your reproducible reports.

We will focus on learning how to configure your document so that it:

  1. Manages its own software dependencies, installing and loading the R packages it needs (Install R Dependencies and Load Packages).
  2. Sets global parameters for knitting, so that every code chunk behaves consistently (YAML Metadata Section).
  3. Produces an appendix of software citations, generated from the packages actually used (Generate a Citation File and Generate Appendix with Citations).
  4. Produces an appendix of package versions and operating system details, recording the environment your results came from (Generate Appendix with Versions and OS).

3.3.3 How These Tutorials Are Written

Every section of this tutorial, and of PART A before it, follows the same three-part protocol. The structure is deliberate:

  1. Why and what. Each section opens by explaining what the skill is and why it matters for reproducibility, before any code appears. A procedure you can follow but cannot justify is one you will drop the first time it becomes inconvenient.
  2. A step-by-step procedure. The implementation itself, numbered, in the order you carry it out, with the code and the options it needs.
  3. A ✅ Checkpoint. What your file and your knitted output should look like once the section is done, so that you can confirm you are on track before building the next piece on top of it.

This is the format I am asking you to use for your own bioinformatics tutorial. Your peers will work through your chapter exactly as you are working through this one: they need to know why they are doing something, precisely how to do it, and how to tell whether it worked. As you go, notice which explanations help you and which leave you stuck, and build that into what you write.

3.3.4 Structure of an R Markdown Document (10 minutes)

3.3.4.1 What You Are Building

What you are building. By the end of PART B you will have one R Markdown document that works as a template for every reproducible report you write afterwards: it installs its own dependencies, formats its citations to a journal style, and ends with appendices listing the software it used. You will not be writing research content into it. You are building the scaffolding that research content later sits inside.

If you want to see where this is going, the finished document is available as Chapter_2_partB_template.Rmd. It is the end state, not a shortcut: work through the sections yourself and use it to check your own file against.

3.3.4.2 How We Will Work

How we will work. This runs like PART A. You build up a single .Rmd file section by section, in the order given, and knit after every section so that a problem is always traceable to the last thing you changed. Each section ends with a ✅ Checkpoint telling you what your file and your knitted output should look like at that point: open it before moving on. Work at your own pace and ask me as soon as something does not knit.

3.3.4.3 The Structure of the Document

To support the reproducibility of your research, we structure the R Markdown file as follows (Figure 3.10):

A stacked block diagram of an .Rmd file read top to bottom: a green YAML metadata block, then blue code chunks for packages and setup, a white YOUR RESEARCH block, and headings for References and Appendices followed by two further blue code chunks for software citations and package versions; arrows on the right connect four blocks to notes about declaring .bib and .csl files, loading and analyzing data, automatic formatting of references, and using packages.bib to build the software bibliography.

Figure 3.10: Representation of the RMarkdown file structure taught in this class. The following colors represent the three main computing languages found in an RMarkdown document: Black: Markdown (we also sometimes use HTML), Green: YAML, and Blue: R (included in code chunks). See text for more details.

  1. The YAML metadata section (at the top of the document) must include information about your bibliography (*.bib) and citation style language (*.csl) files, both of which must be stored in the working directory. (YAML Metadata Section)
  2. Two R code chunks are provided just below the YAML metadata section:
  3. The Markdown section beneath the two R code chunks is edited to add headers for the References and Appendices sections. (Generate Appendix with Citations)
  4. Finally, two R code chunks are included in the Appendices section:

Three of the four code chunks named above are what you build in this tutorial, one at a time: chunk #1 in Install R Dependencies and Generate a Citation File, chunk #3 in Generate Appendix with Citations, and chunk #4 in Generate Appendix with Versions and OS. Code chunk #2, which sets global options for code chunks, comes later in PART D.

3.3.4.4 This Page Is Built the Same Way

The page you are reading is built exactly this way. These course chapters are themselves one large R Markdown document, and it follows the structure in Figure 3.10 point for point. Open Chapters.Rmd on GitHub and you will find, in order:

  • a YAML metadata section declaring csl: AmJBot.csl and two bibliography files, one of which is packages.bib;
  • a packages chunk with echo = FALSE, warning = FALSE, include = FALSE, which installs and loads the dependencies and ends with knitr::write_bib(.packages(), file = 'packages.bib');
  • a setup chunk setting the global options for every code chunk (code chunk #2, the one you meet in PART D);
  • all of the course content, which is the your research block in the figure;
  • and at the very bottom, a # References header, <div id="refs"></div>, and Appendix 1 and Appendix 2, generated from packages.bib and sessionInfo().

Scroll to the end of this page and you can read those two appendices as output, then read the code that produced them on GitHub. Nothing in this course was written by hand and described as reproducible. The chapters, the syllabus and the schedule are all produced by the workflow you are being taught, which is the only honest way to teach it.

3.3.5 Create Your R Markdown Document (10 minutes)

Use the material from the Chapter 2 – Part A and Before Starting sections to create and save your R Markdown document for this tutorial.

💡 Guidelines

  • Document title: Chapter 2 - part B
  • Document output format: HTML
  • File name: Chapter2_partB.Rmd
  • File path: Reproducible_Science/Chapters/Chapter_2

✅ Checkpoint

Your project folder contains Chapter2_partB.Rmd, saved at Reproducible_Science/Chapters/Chapter_2.

Your document knits to an HTML file without error, and the .html appears alongside the .Rmd.

If it does not knit yet, fix that before going any further. Everything below is added to this file.

3.3.6 YAML Metadata Section (25 minutes)

The YAML metadata section (Figure 3.10) allows users to provide arguments (referred to as fields) that control how an R Markdown document is converted into its final output format. YAML is a small, general-purpose language for writing structured settings in plain text, used well beyond R Markdown; the anatomy of an R Markdown document shows how it fits into the rest of the file. In this class, we will use functions from the knitr (Xie, 2015, 2025b) and bookdown (Xie, 2016, 2025a) packages to populate this section. (Field names, as declared in the YAML metadata section, are provided in parentheses.)

  1. Title (title)
  2. Subtitle (subtitle)
  3. Author(s) (author)
  4. Date (date)
  5. Output format(s) (output)
  6. Citations link (link-citations)
  7. Font size (fontsize)
  8. Bibliography file(s) (bibliography)
  9. Citation style file for journal formatting (csl)

The YAML code shown below generates either an HTML or PDF document (see the output field), includes a table of contents (see the toc field), and formats in-text citations and the bibliography section according to the journal style specified in the AmJBot.csl file (via the csl field). The bibliography must be stored in a .bib file (in this case, Bibliography_Reproducible_Science_2.bib) placed at the root of your working directory. For this chapter that folder is Reproducible_Science/Chapters/Chapter_2, the same folder as the Chapter2_partB.Rmd file you created in Create Your R Markdown Document. The AmJBot.csl file goes in there too: both files must sit beside the .Rmd, not in a subfolder.

--- 
title: "Your title"
subtitle: "Your subtitle"
author: "Your name"
date: "`r Sys.Date()`"
output:
  bookdown::html_document2: 
    toc: TRUE
  bookdown::pdf_document2:
    toc: TRUE
link-citations: yes
fontsize: 12pt
bibliography: Bibliography_Reproducible_Science_2.bib 
csl: AmJBot.csl
---

3.3.6.1 Step-by-Step Procedure

Follow these steps to set up your YAML metadata section (also see Figure 3.10):

  1. Make sure you have created a document following the approach described here
  2. Copy the following files, available on Google Drive, into your project folder:
    • Bibliography_Reproducible_Science_2.bib
    • AmJBot.csl
  3. Replace your YAML metadata section (at the top of your R Markdown document) with the code as shown in the section above
  4. Knit your document and inspect the output file. Read this section to get more details on the knitting procedure.

💡 Info

  • Warning: The .bib and .csl files must be stored in the same working directory as your .Rmd file.
  • R functions in the YAML metadata section: This can be done using inline R code syntax. For example, use Sys.Date() to automatically add the current date to the output document.
  • Bibliography files: You can use multiple bibliography files by specifying them like this: bibliography: [file1.bib, file2.bib].
  • Omitting parameters in the YAML metadata section: You can omit parameters by commenting them out with a # at the beginning of the line (equivalent to commenting).
  • More information: To learn more about YAML and its use in R Markdown, visit this website.

3.3.6.2 Knitting Procedure

Since you have declared two output formats in the YAML metadata section, and both are specific to bookdown functions, you need to select which format you want to use to compile your document. To do this, click the drop-down menu to the left of the Knit button (see Figure 3.11).

To ensure that bookdown functions are applied correctly, make sure to select one of the following options (see Figure 3.11):

  • Knit to html_document2
  • Knit to pdf_document2
Screenshot of the RStudio source editor with the Knit button drop-down menu open at the top left, listing Knit to html_document2, Knit to pdf_document2, Knit with Parameters, Knit Directory and Clear Knitr Cache.

Figure 3.11: Snapshot of RStudio console showing the drop-down list associated to Knit button.

✅ Checkpoint

Your .Rmd begins with a YAML block that ends csl: AmJBot.csl.

Your project folder also contains Bibliography_Reproducible_Science_2.bib and AmJBot.csl, beside the .Rmd file.

Your knitted output opens with your title, author and today’s date, followed by a table of contents.

If the document will not knit, the most common cause at this stage is one of those two files being in the wrong folder, or a stray indent in the YAML.

3.3.7 Install R Dependencies and Load Packages (25 minutes)

3.3.7.1 Introduction

It is best practice to add an R code chunk directly below the YAML metadata section to automatically install and load the required R packages.

This approach offers two key benefits that support the reproducibility of your research:

  1. It ensures that all dependencies needed to produce your document are installed and available (see Figure 3.10).
  2. It allows you to automatically generate a citation file for all R packages used in your report (see below).

💡 Disclaimer

  • Warning: The code provided in this section will install the following R packages on your computer: “knitr”, “rmarkdown”, “bookdown”, “formattable”, “kableExtra”, “dplyr”, “magrittr”, “prettydoc”, “htmltools”, “knitcitations”, “bibtex”, “devtools”.
  • More on the R packages: These packages are required to produce your reproducible reports in RStudio. All of them are available on CRAN, the official repository for R.
  • R package repositories: The code below sets the CRAN mirror for you, with repos = “https://cloud.r-project.org/”, so the install cannot stop to ask you which mirror to use. That matters when knitting, because a prompt has nowhere to appear and the document simply fails. If you would rather set a default mirror in RStudio as well, follow this procedure.
  • Questions or concerns: If you have any questions, please contact the instructor at svenbuerki@boisestate.edu.

3.3.7.2 Step-by-Step Procedure

  1. Insert an R code chunk directly below your YAML metadata section (see Figure 3.10).
  2. Name the code chunk packages and set the chunk options as follows (see Figure 3.12):
    • echo = FALSE
    • warning = FALSE
    • include = FALSE
      Close-up of a single R Markdown chunk header that opens with three backticks and reads r packages, echo=FALSE, warning=FALSE, include=FALSE, followed by the closing backticks on the next line.

      Figure 3.12: This is how your packages R code chunk should look like at this stage of the procedure.

  3. Copy the following code into your R code chunk (Figure 3.12) to install and load the packages required to produce your report:
###~~~
# Load R packages
###~~~
#Create a vector w/ the required R packages
# --> If you have a new dependency, don't forget to add it in this vector
pkg <- c("knitr", "rmarkdown", "bookdown", "formattable", "kableExtra", "dplyr", "magrittr", "prettydoc", "htmltools", "knitcitations", "bibtex", "devtools")

##~~~
#2. Check if pkg are already installed on your computer
##~~~
print("Check if packages are installed")
#This line outputs a list of packages that are not installed
new.pkg <- pkg[!(pkg %in% installed.packages())]

##~~~
#3. Install missing packages
##~~~
# Use an if/else statement to check whether packages have to be installed
# WARNING: If your target R package is not deposited on CRAN then you need to adjust code/function
if(length(new.pkg) > 0){
  print(paste("Install missing package(s):", new.pkg, sep=' '))
  # repos= names the CRAN mirror explicitly, so the install never stops to ask
  # which mirror to use. This matters when knitting: a prompt has nowhere to
  # appear and the document fails. cloud.r-project.org is CRAN's global mirror.
  install.packages(new.pkg, dependencies = TRUE, repos = "https://cloud.r-project.org/")
}else{
  print("All packages are already installed!")
}

##~~~
#4. Load all required packages
##~~~
print("Load packages and return status")
#Here we use the sapply() function to require all the packages
# To know more about the function type ?sapply() in R console
sapply(pkg, require, character.only = TRUE)
  1. Review the R code to ensure you understand what it does and why
  2. Knit your document and inspect the output file
✅ Checkpoint

Directly below your YAML there is a code chunk named packages, with the options echo = FALSE, warning = FALSE and include = FALSE.

The R code from step 3 is inside that chunk, not just referenced from it: the pkg vector listing every package, the check for missing packages, the if statement that installs them, and the sapply(pkg, require, character.only = TRUE) line that loads them. A chunk with the right name and the right options but no code in it will knit perfectly and do nothing at all.

When you knit, the R Markdown console reports the packages being checked and loaded, but nothing from this chunk appears in the output document. That is what those three options are for.

If a package fails to install, stop and ask rather than removing it from the vector.

3.3.8 Generate a Citation File of R Packages Used (12 minutes)

3.3.8.1 Introduction

I do not know about you, but I often struggle to properly cite R packages in my publications. Fortunately, R provides a built-in function to help with this. If you want to retrieve the citation for an R package, you can use the base R function citation().

For example, the citation for the knitr package can be obtained as follows:

# Generate citation for knitr Type this code directly in
# the Console
citation("knitr")
## To cite package 'knitr' in publications use:
## 
##   Xie Y (2025). _knitr: A General-Purpose Package for Dynamic Report
##   Generation in R_. R package version 1.50, <https://yihui.org/knitr/>.
## 
##   Yihui Xie (2015) Dynamic Documents with R and knitr. 2nd edition.
##   Chapman and Hall/CRC. ISBN 978-1498716963
## 
##   Yihui Xie (2014) knitr: A Comprehensive Tool for Reproducible
##   Research in R. In Victoria Stodden, Friedrich Leisch and Roger D.
##   Peng, editors, Implementing Reproducible Computational Research.
##   Chapman and Hall/CRC. ISBN 978-1466561595
## 
## To see these entries in BibTeX format, use 'print(<citation>,
## bibtex=TRUE)', 'toBibtex(.)', or set
## 'options(citation.bibtex.max=999)'.

If you want to generate those citation entries in BibTeX format, you can pass the object returned by citation() to toBibtex() as follows:

# Generate citation for knitr in BibTex format Note that
# there is no citation identifiers. Those will be
# automatically generated in our next code.
utils::toBibtex(utils::citation("knitr"))
## @Manual{,
##   title = {knitr: A General-Purpose Package for Dynamic Report Generation in {R}},
##   author = {Yihui Xie},
##   year = {2025},
##   note = {R package version 1.50},
##   url = {https://yihui.org/knitr/},
## }
## 
## @Book{,
##   title = {Dynamic Documents with {R} and knitr},
##   author = {Yihui Xie},
##   publisher = {Chapman and Hall/CRC},
##   address = {Boca Raton, Florida},
##   year = {2015},
##   edition = {2nd},
##   note = {ISBN 978-1498716963},
##   url = {https://yihui.org/knitr/},
## }
## 
## @InCollection{,
##   booktitle = {Implementing Reproducible Computational Research},
##   editor = {Victoria Stodden and Friedrich Leisch and Roger D. Peng},
##   title = {knitr: A Comprehensive Tool for Reproducible Research in {R}},
##   author = {Yihui Xie},
##   publisher = {Chapman and Hall/CRC},
##   year = {2014},
##   note = {ISBN 978-1466561595},
## }

💡 Info

  • Note that both citation() and toBibtex() are functions in the utils R package.
  • To make sure that you are calling the right R function, you can add the package name in front of the function, followed by ::. In this example, the code is as follows: utils::toBibtex(utils::citation(“knitr”)).

3.3.8.2 Step-by-Step Procedure

Here, you will edit the packages code chunk to output a .bib file containing references for all R packages used to generate your document. This file will be included in the appendix 1 presented in the next section.

This can be done by adding the following code at the end of your packages R code chunk:

# Generate BibTex citation file for all loaded R packages
# used to produce report Notice the syntax used here to
# call the function
knitr::write_bib(.packages(), file = "packages.bib")

The .packages() function returns invisibly the names of all packages loaded in the current R session (to see the output, use .packages(all.available = TRUE)). This ensures that all packages used in your code will have their citation entries written to the .bib file.

Finally, to be able to cite these references (see Citation identifier) in your text, you need to edit the YAML metadata section. See Appendix 1 for a full list of references associated with the R packages used to generate this report.

💡 Your turn to practice this syntax in your document by:

  • Adding the code to generate packages.bib in your R code chunk entitled packages.
  • Knitting your document and inspecting your project folder to observe the creation of a new file entitled packages.bib.
✅ Checkpoint

Your packages chunk now ends with knitr::write_bib(.packages(), file = 'packages.bib').

After knitting, a new file packages.bib has appeared in your project folder. Open it: it holds a BibTeX entry for every package loaded by the chunk above.

No packages.bib means the line is outside the chunk, or the document has not been knitted since you added it.

3.3.9 Generate Appendix with Citations of Used R Packages (12 minutes)

Although a References section will be provided at the end of your document to cite in-text references (see References and Figure 3.10), it is useful to create a customized appendix listing citations for all R packages used to conduct the research, as shown in Appendix 1.

Here, we will learn the procedure to assemble such an appendix.

3.3.9.1 Step-by-Step Procedure

  1. Add a level-1 header at the end of your document titled “References”.
  2. Include the appendices after the References section (see Figure 3.10) by following the steps below:
    • Insert <div id="refs"></div> as shown below. This ensures that the appendices (or any other material) appear after the References section.
      (See this resource for more details.)
# References

<div id="refs"></div>

# (APPENDIX) Appendices {-}

# Appendix 1

Citations of all R packages used to generate this report.
  1. Insert an R code chunk directly below # Appendix 1 to read and print the citations saved in packages.bib. Use the following code:
### Load R package
library("knitcitations")
### Process and print citations in packages.bib Clear all
### bibliography that could be in the cash
cleanbib()
# Set pandoc as the default output option for bib
options(citation_format = "pandoc")
# Read and print bib from file
read.bibtex(file = "packages.bib")
  1. Edit your R code chunk options as follows to correctly print the references:
    {r generateBibliography, results = "asis", echo = FALSE, warning = FALSE, message = FALSE}
  2. Knit your document to check that it produces the correct output (see Knitting procedure). Refer to Appendix 1 to see an example of the expected result.
✅ Checkpoint

Your document now ends with, in this order: a # References header, <div id="refs"></div>, # (APPENDIX) Appendices {-}, # Appendix 1, and a chunk named generateBibliography carrying results = "asis".

In the knitted output, Appendix 1 appears after the References section and lists the citations of every R package you loaded.

If the appendix appears before the references, the <div id="refs"></div> line is missing or misplaced.

3.3.10 Generate Appendix with R Package Versions and Operating System (8 minutes)

In addition to citing R packages, you may also want to provide detailed information about the R package versions and operating system used (see Figure 3.10).

In R, the simplest — yet highly useful and important — way to document your environment is to include the output of sessionInfo() (or devtools::session_info()). Among other details, this output shows all loaded packages and their versions from the R session used to run your analysis.

Providing this information allows others to reproduce your work more reliably, as they will know exactly which packages, versions, and operating system were used to execute the code.

For example, here is the output of sessionInfo() showing the R version and packages I used to create this document:

# Collect Information About the Current R Session
sessionInfo()
## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Sequoia 15.7.9
## 
## Matrix products: default
## BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
## 
## locale:
## [1] en_US/en_US/en_US/C/en_US/en_US
## 
## time zone: America/Chicago
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] klippy_0.0.0.9500    rticles_0.27         DiagrammeR_1.0.11   
##  [4] DT_0.34.0            data.tree_1.2.0      kfigr_1.2.1         
##  [7] devtools_2.4.6       usethis_3.2.1        bibtex_0.5.1        
## [10] knitcitations_1.0.12 htmltools_0.5.9      prettydoc_0.4.1     
## [13] magrittr_2.0.4       dplyr_1.2.1          kableExtra_1.4.0    
## [16] formattable_0.2.1    bookdown_0.46        rmarkdown_2.30      
## [19] knitr_1.50          
## 
## loaded via a namespace (and not attached):
##  [1] xfun_0.54          bslib_0.9.0        htmlwidgets_1.6.4  remotes_2.5.0     
##  [5] visNetwork_2.1.4   vctrs_0.7.3        tools_4.5.2        crosstalk_1.2.2   
##  [9] generics_0.1.4     tibble_3.3.0       RefManageR_1.4.0   pkgconfig_2.0.3   
## [13] RColorBrewer_1.1-3 assertthat_0.2.1   lifecycle_1.0.5    compiler_4.5.2    
## [17] farver_2.1.2       stringr_1.6.0      textshaping_1.0.4  sass_0.4.10       
## [21] yaml_2.3.12        pillar_1.11.1      jquerylib_0.1.4    ellipsis_0.3.2    
## [25] cachem_1.1.0       magick_2.9.0       sessioninfo_1.2.3  tidyselect_1.2.1  
## [29] digest_0.6.39      stringi_1.8.9      purrr_1.2.2        fastmap_1.2.0     
## [33] cli_3.6.5          pkgbuild_1.4.8     scales_1.4.0       backports_1.5.0   
## [37] lubridate_1.9.4    timechange_0.3.0   httr_1.4.7         memoise_2.0.1     
## [41] evaluate_1.0.5     viridisLite_0.4.2  rlang_1.3.0        Rcpp_1.1.0        
## [45] glue_1.8.0         formatR_1.14       xml2_1.5.1         pkgload_1.4.1     
## [49] svglite_2.2.2      rstudioapi_0.17.1  jsonlite_2.0.0     R6_2.6.1          
## [53] plyr_1.8.9         systemfonts_1.3.1  fs_1.6.6

3.3.10.1 Step-by-Step Procedure

I have also used the approach described above to add this information in Appendix 2. This can be done as follows:

  1. Edit your document to add the following text below Appendix 1.
# Appendix 2

Version information about R, the operating system (OS) and attached or R loaded packages. This appendix was generated using `sessionInfo()`.
  1. Then, add an R code chunk and set its options as follows to correctly print the content:
    {r sessionInfo, eval = TRUE, echo = FALSE, warning = FALSE, message = FALSE}
  2. Copy this code in your code chunk:
# Load and provide all packages and versions
sessionInfo()
  1. Knit your document to check that it produces the correct output.
✅ Checkpoint

Your document ends with # Appendix 2, its one-sentence description, and a chunk named sessionInfo set to eval = TRUE, echo = FALSE.

In the knitted output, Appendix 2 lists your R version, your operating system, and every attached package with its version number.

That output is specific to your machine, so it will not match mine, and that is precisely the point: it records the environment your results were produced in.

3.3.11 All Set — You’re Good to Go!

You have now set up your R Markdown environment and are ready to start populating it! This means you can begin inserting your text and additional code chunks directly below the packages code chunk.

Compare your document against the finished template, Chapter_2_partB_template.Rmd. It contains the YAML metadata section, the packages chunk (including the line that writes packages.bib), the References section with <div id="refs"></div>, and both appendices, in the order the figure above shows. If something in your own file is missing or out of order, this is where to look.

The References section marks the end of the main body of your document. If you wish to add appendices, do so under Appendix 2. Note that appendices will be labeled differently from the main sections of the document.

💡 Info

  • This R Markdown document will be used as a template for the rest of Chapter 2.
  • This document also serves as a foundation for your Bioinformatics Tutorial.

3.4 PART C: Tables, Figures and References

3.4.1 Aim

In PART B you built a template. The aim of this tutorial is to fill it with the elements a scientific report is actually made of: tables, figures and references.

All of this is done with bookdown (Xie, 2016, 2025a) and related R packages (see Appendix 1 for a full list). Unlike the functions implemented in the R rmarkdown package (Xie et al., 2018, which is better suited for generating PDF reproducible reports), bookdown provides one unified set of functions that generates both HTML and PDF documents, and that same set handles tables, figures and the cross-references pointing at them.

We will focus on learning how to:

  1. Generate tables and figures from code, so that they are rebuilt every time the document is knitted instead of being pasted in by hand (Insert Tables and Insert Figures).
  2. Cross-reference them in the text, so that the numbering stays correct when you add, remove or reorder content (Cross-reference Tables and Figures in the Text).
  3. Cite references from a BibTeX file, using citation identifiers rather than typing author names and years (Cite References in the Text).
  4. Generate the References section automatically, from the citations actually present in your text (Generate a References Section).
  5. Format citations to a journal style, by declaring a CSL file in the YAML metadata section (Format Citations to Journal Style).

3.4.2 Learning Outcomes

This tutorial provides students with the opportunity to learn how to:

We have half a class and the homework that follows it for this, and each outcome above maps to one section below. The times are a pace rather than a deadline, in the same way as in PART A and PART B:

Block Time Where
Create Your R Markdown Document 5 min In class
Insert Tables 23 min In class
Insert Figures 20 min At home
Insert a Figure from an Image File 20 min In class, 1 Oct
Cross-reference Tables and Figures in the Text 10 min At home
Cite References in the Text 15 min At home
Generate a References Section 10 min At home
Format Citations to Journal Style 10 min At home
Total 113 min

3.4.3 How This Tutorial Is Written

PART C follows the same protocol as PART B. Each section opens with why and what, gives a step-by-step procedure, and follows that procedure with a ✅ Checkpoint describing what your file and your knitted output should look like before you build the next piece on top of them.

PART C adds one more element, and it is not a checkpoint. Most sections also carry a 💡 Challenge. A Checkpoint confirms that what the tutorial told you to do actually worked. A Challenge asks you to go past what the tutorial showed you: to work out why a piece of syntax behaves as it does, to test that reasoning by editing your own document and knitting it, or to go and find something nobody handed you. The procedure gets your document working. The Challenge is where you start owning the tool, which is the part that transfers to your own research.

Every Challenge has a ✅ Solution folded underneath it, in the same way as in PART A. Open it once you have made a real attempt, not before. The answer is worth far less than the ten minutes you spend not having it, and on your own data there will be no toggle to open.

Knit after every section, so that a problem is always traceable to the last thing you changed. This is also the format I am asking you to use for your own bioinformatics tutorial.

3.4.4 Create Your R Markdown Document (5 minutes)

You are not starting from a blank file. The document you finished in PART B already knows how to install its dependencies, where to find your bibliography, and how to build its own appendices, so PART C begins by making a copy of it.

Follow the approach described in the box below to create your R Markdown file for Chapter 2 – Part C.

💡 Approach to Create Your R Markdown

  • Create document: Make a copy of Chapter2_partB.Rmd in the same project folder and rename it Chapter2_partC.Rmd
  • Download document: If you have any issues with Chapter 2 - part B, then download the instructor version on Google Drive
  • Document title: Open the file in RStudio and edit the title in the YAML section to Chapter 2 – Part C
  • Test document: Knit your document to ensure it works properly
✅ Checkpoint

Your project folder contains Chapter2_partC.Rmd, sitting next to Chapter2_partB.Rmd at Reproducible_Science/Chapters/Chapter_2.

Its YAML title reads Chapter 2 - Part C. Everything else in the YAML is unchanged from PART B, including the bibliography: and csl: fields, which you will rely on later in this tutorial.

It knits to HTML without error, and the output already contains a References section and both appendices. If it knits, PART B is behind you and everything below is an addition to a document that already works.

3.4.5 Insert Tables (23 minutes)

This tutorial introduces key concepts related to table creation in R Markdown, specifically the following:

  • Creating a table based on R code
  • Assigning a table caption
  • Providing a unique label to the R code chunk for cross-referencing in the text
  • Displaying the table in the document

More details on this topic will be provided in Chapter 9.

In this section, you will learn the R Markdown syntax and R code needed to replicate the grading scale presented in the syllabus (see Table 3.2).

Table 3.2: Grading scale applied in this class.
Percentage Grade
100-98 A+
97.9-93 A
92.9-90 A-
89.9-88 B+
87.9-83 B
82.9-80 B-
79.9-78 C+
77.9-73 C
72.9-70 C-
69.9-68 D+
67.9-60 D
59.9-0 F

3.4.5.1 The R Packages and Syntax Used Here (~3 minutes)

Before you type the code, it is worth knowing what is doing the work in this line:

knitr::kable(grades, caption = "Grading scale applied in this class.",
    format = "html") %>%
    kableExtra::kable_styling(c("striped", "scale_down"))

💡 The R packages used here

  • knitr provides kable(), which turns an R object (here a data frame) into a formatted table.
  • kableExtra provides kable_styling(), which controls how that table looks: striped rows, scaling to fit the page, and so on.
  • You installed both in Install R Dependencies and Load Packages, so there is nothing new to install. This is the payoff for building that packages chunk in PART B.

💡 Two pieces of syntax in that line

  • package::function() calls a function and says which package it came from. knitr::kable() means the kable() function in knitr. Once a package is loaded you can write kable() on its own, but naming the package makes the code unambiguous, and it keeps working if another package you load later also defines a kable().
  • %>% is the pipe, and it is not part of base R. It is defined in the magrittr package and re-exported by dplyr and kableExtra, which is exactly why the chunk begins with library(dplyr): kableExtra::kable_styling() uses ::, and :: does not attach a package, so without that library() call the pipe would not exist. It takes whatever is on its left and hands it to the function on its right as that function’s first argument, so x %>% f() is another way of writing f(x).

💡 There are two pipes in R

  • %>% comes from magrittr, and is re-exported by dplyr, kableExtra and the rest of the tidyverse. It only works when one of those packages is attached, so any document using it carries that dependency.
  • |> is R’s native pipe, added in R 4.1.0 (2021). It needs no package at all, because it is part of the language.

That difference is a reproducibility question, not a style one: x |> f() runs on any R 4.1 or later, while x %>% f() fails on a machine where magrittr is not installed. These materials use %>% because most published R code still does and you will need to read it, but |> is a perfectly good choice in your own work, and one fewer thing for a reader to install.

💡 The same table, with and without the pipe 

# with the pipe: build the table, then style it
knitr::kable(grades) %>% kableExtra::kable_styling("striped")

# without the pipe: the same thing, read from the inside out
kableExtra::kable_styling(knitr::kable(grades), "striped")

Both produce the same table. The pipe earns its keep as soon as there are three or four steps, because it keeps them in the order you actually do them instead of nesting them inside one another. You will meet it constantly in R code.

3.4.5.2 Step-by-Step Procedure (~10 minutes)

Follow these steps to reproduce Table 3.2 in your R Markdown document:

  1. Open Chapter2_partC.Rmd in RStudio.
  2. Add a first-level header titled Tables below the packagescode chunk.
  3. Insert an R code chunk under your header by clicking the Insert button in the editor toolbar.
  4. Copy and paste the following R code into your code chunk:
### Load package (for testing)
library(dplyr)
### Create a data.frame w/ grading scale
grades <- data.frame(Percentage = c("100-98", "97.9-93", "92.9-90",
    "89.9-88", "87.9-83", "82.9-80", "79.9-78", "77.9-73", "72.9-70",
    "69.9-68", "67.9-60", "59.9-0"), Grade = c("A+", "A", "A-",
    "B+", "B", "B-", "C+", "C", "C-", "D+", "D", "F"))
### Plot table and add caption format = 'html' tells
### kable() which format to produce. Leaving it out is what
### causes a caption to come out as 'Table Table 1:' (see
### the note below).
knitr::kable(grades, caption = "Grading scale applied in this class.",
    format = "html") %>%
    kableExtra::kable_styling(c("striped", "scale_down"))
  1. Edit the R code chunk options line by adding the following argument (Note: each argument should be separated by a comma):
    • echo = FALSE
  2. Add the unique label tabgrades in the chunk options line (immediately after {r) to enable cross-referencing.
  3. Test your R code to ensure it produces the expected table by clicking the Play button at the top left of the code chunk (see Fig. 3.13).
    An R Markdown code chunk with a green Run arrow highlighted at the top right, and below it the inline preview pane showing the resulting grading scale table with Percentage and Grade columns starting at 100-98 for A+.

    Figure 3.13: Demonstrates how to test your code within the R Markdown document. Click the play icon in the top-right corner of the code chunk to execute the code and return its output directly in the R Markdown document without knitting.

  4. Knit your document using the Knit button in the editor toolbar (see Figure 3.11) to inspect the output and check your syntax and code.

💡 If your caption comes out as “Table Table 1:”

The format = “html” argument in the code above is what prevents this. Without it, kableExtra re-reads the Markdown table that kable() produced and treats it as a new table, adding a second “Table” prefix.

If you see the doubled caption anyway, replace that last line with kableExtra::kbl(), which sets the format by itself: 

kableExtra::kbl(grades, caption = "Grading scale applied in this class.") %>%
  kableExtra::kable_styling(c("striped", "scale_down"))

Knit again and the caption should read “Table 1:” once.

✅ Checkpoint

Your document has a first-level Tables header below the packages code chunk, and under it a code chunk labelled tabgrades carrying the option echo = FALSE.

The whole block from step 4 is inside that chunk: the library(dplyr) call, the grades data frame, and the knitr::kable() call with both its caption argument and format = "html".

In the knitted output you see a striped table of percentages and letter grades, captioned Table 1: Grading scale applied in this class. You did not type that number: bookdown assigned it, which is why the numbering will still be right after you add a second table.

If your caption reads Table Table 1:, see the note just above.

3.4.5.3 Challenge (~10 minutes)

💡 Is something missing?

  • Challenge: Scrutinize the code provided in this section to determine how the table caption is created.
  • Test Your Hypothesis: Once you have figured out how the caption is assigned, test your hypothesis by editing it and knitting your document.
✅ Solution

The caption is the caption argument of knitr::kable(), inside the code chunk:

knitr::kable(grades, caption = "Grading scale applied in this class.", format = "html")

Edit that string, knit, and the caption changes. Two things are worth noticing while you do it:

  • You never type the number. The words Table 1: are added by bookdown when it knits, which is why inserting a new table above this one renumbers everything correctly instead of leaving you to fix it by hand.
  • The caption belongs to the code, not to the text. It lives in the same chunk that builds the table, so a table can never end up separated from its caption.

3.4.6 Insert Figures (20 minutes)

This tutorial introduces key concepts related to figure creation in R Markdown, specifically the following:

  • Creating a figure based on R code.
  • Assigning a figure caption.
  • Providing a unique label to the R code chunk allowing further cross-referencing in the text.
  • Displaying the figure in the document.

More details on this topic will be provided in Chapter 10.

In this section, you will learn the R Markdown syntax and R code needed to replicate Figure 3.14.

Plot of cars' speed in relation to distance.

Figure 3.14: Plot of cars’ speed in relation to distance.

3.4.6.1 Step-by-Step Procedure (~10 minutes)

Follow these steps to reproduce Figure 3.14 in your R Markdown document:

  1. If not done yet, open Chapter2_partC.Rmd in RStudio.
  2. Add a first-level header titled Figures below the Tables header.
  3. Insert an R code chunk under your header by clicking the Insert button in the editor toolbar.
  4. Copy and paste the following R code into your code chunk:
### Load and summarize the cars dataset
summary(cars)
### Plot data
plot(cars)
  1. Edit the R code chunk options line by adding the following argument (Note: each argument should be separated by a comma):
  • echo = FALSE
  • results = "hide"
  • fig.cap = "Plot of cars' speed in relation to distance."
  • out.width = "100%"
  1. Add the unique label cars in the chunk options line (immediately after {r) to enable cross-referencing.
  2. Test your R code to ensure it produces the expected table by clicking the Play button at the top left of the code chunk (as done in Tables).
  3. Knit your document using the Knit button in the editor toolbar (see Figure 3.11) to inspect the output and check your syntax and code.
✅ Checkpoint

Your document has a first-level Figures header below the Tables section, and under it a code chunk labelled cars.

Its options include echo = FALSE, results = "hide", fig.cap and out.width, each separated from the next by a comma.

In the knitted output the scatter plot of speed against distance spans the full width of the page and is captioned Figure 1: Plot of cars’ speed in relation to distance.

The numeric output of summary(cars) does not appear anywhere. The code still runs, but results = "hide" keeps its printed output out of the document. If you can see a block of numbers above your figure, that option is missing or misspelled.

3.4.6.2 Challenge (~10 minutes)

💡 Challenge

  • What syntax is used to assign a figure caption?
  • How does the syntax for assigning captions to tables and figures compare?
✅ Solution

A figure caption is a chunk option, fig.cap, written in the chunk header:

```{r cars, echo = FALSE, results = "hide", fig.cap = "Plot of cars' speed in relation to distance.", out.width = '100%'}

A table caption is a function argument, caption, written inside the chunk body:

knitr::kable(grades, caption = "Grading scale applied in this class.", format = "html")

Why they differ. A figure is produced by the chunk as a whole: knitr catches whatever plot() draws and wraps it, so the caption has to be declared on the chunk. A table is just the value returned by kable(), so its caption is one of that function’s arguments.

What is the same, and it is the part that matters. Both are numbered by bookdown, and both are cross-referenced through the chunk label (tab:tabgrades, fig:cars). That is also why a table and a figure you want to reference separately need to be in separate chunks: the label is attached to the chunk, not to the output.

3.4.7 Insert a Figure from an Image File (20 minutes)

3.4.7.1 Goal

There are two ways a figure gets into an R Markdown report, and so far you have only met one.

In Insert Figures the figure was generated: plot(cars) drew it while the document knitted, and knitr caught whatever the chunk produced. No image existed before you pressed Knit, and none is left behind that you put there yourself.

Many figures in a real report are the other kind. They are files that already exist before you start writing: a photograph, a diagram you drew by hand, a map exported from GIS, a plot produced by software that is not R. Nothing draws them at knit time, so instead of running code that makes a picture, your chunk has to tell knitr where the picture already lives.

The goal of this section is to do exactly that. By the end you will have put an existing .png into your document, captioned it, numbered it and cross-referenced it, so that it behaves like every other figure in the report.

3.4.7.2 How We Will Do It

You are going to work this one out rather than be walked through it, and you have a good source to work from: the course syllabus is an R Markdown document exactly like yours, and one of its figures is an image file. The code you need is therefore already written, already working, and published where you can read it.

So the method is: find that chunk, understand what each part of it does, and adapt it to your own document.

That last word is the whole exercise. The code cannot be copied across unchanged, because the syllabus and your document sit in different folders, and a path that is correct in one is wrong in the other. Working out what has to change, and why, is the part of this that will still be useful to you long after you have forgotten the function name.

💡 Before you start

  • There is no step-by-step procedure in this section. The ✅ Solution is where the explanation lives, so open it after you have had a real attempt, not before.
  • Work in Chapter2_partC.Rmd, the document you built in PART C.
  • The syntax is not the hard part. The path is. Read the whole challenge before you start typing.

3.4.7.3 Challenge (~20 minutes)

💡 Replicate a figure from the syllabus

Figure 7.1 of the syllabus, Overview of workflow studied in PART2, is an image file inserted into an R Markdown document. Reproduce it in your own Chapter2_partC.Rmd.

  • Find the code. The syllabus is an R Markdown document exactly like yours. Open its source on GitHub, index.Rmd, and find the chunk that produces that figure.
  • Get the image. Open Bioinformatic_workflow_PART2.png on GitHub. Because it is an image and not code, GitHub shows you a preview of it rather than text, so there is nothing to copy and paste. Look at the small toolbar above the preview, on the right, and click Download raw file (the ⇩ icon). That hands you the real file under its correct name. Save it, then move it into whichever folder you settle on in the next step, rather than leaving it in Downloads.
  • Put it in a folder of its own. Create a folder called Figures in your Chapter_2 folder, next to Chapter2_partC.Rmd, and save the image there. Keeping images in one named folder rather than loose beside the document is the habit worth forming: it is what makes a project you can hand to somebody else.
  • Work out the path. The path in the syllabus chunk will not work in your document, and understanding why is the point of this exercise. Work out what the path has to be when it is read from your own .Rmd rather than from the repository root, and adjust the code to match.
  • Add a second-level header (##) called Figures from files inside your Figures section. It is a subsection of Figures, not a new top-level one.
  • Insert a code chunk under that header with the Insert button, and write the code that places your image: the function call with your corrected path, and chunk options giving it a unique label, a caption and a width, as you did for the cars figure. Then knit and check the figure appears.
  • Cross-reference it in a sentence, as you did in Cross-reference Tables and Figures in the Text.
✅ Solution

The function is knitr::include_graphics(). It is not a plotting command: you hand it the path to a file and it places that file in your document. The syllabus chunk contains one line:

knitr::include_graphics("Images/Bioinformatic_workflow_PART2.png")

Its chunk header is where the work is:

```{r part2, echo=FALSE, fig.cap="Overview of workflow studied in PART2.", fig.alt="Diagram of a reproducible data analysis workflow. On the left, a box labelled Prepare contains Import followed by Tidy. ...", out.width = '100%', fig.pos= "h"}
  • part2 is the label, so the figure can be cross-referenced as \@ref(fig:part2).
  • echo=FALSE hides the one line of code, which nobody needs to read.
  • fig.cap is the caption, exactly as for the cars figure.
  • out.width = '100%' scales it to the page width.
  • fig.pos = "h" asks for it to stay near this point in the text.

Why the path had to change. index.Rmd sits at the root of the repository, so Images/... is correct relative to it. Your document lives in Reproducible_Science/Chapters/Chapter_2, where no Images folder exists, so copying the path verbatim gives you an error and an empty space where the figure should be. With the image in the Figures folder you created next to your own .Rmd, the path becomes:

knitr::include_graphics("Figures/Bioinformatic_workflow_PART2.png")

The folder name itself is yours to choose. The syllabus happens to call its folder Images, and that is the only reason its path differs from yours: what matters is that the path is read from the folder holding the .Rmd, not that it is spelled a particular way.

This works for the reason you met in Create Your Document and Set Your Working Directory: when knitr knits, the working directory is the folder holding the .Rmd. Every path in your document is read from there, which is also why a figure kept outside that folder makes the document unknittable on anybody else’s machine.

One option you have not met before: fig.alt. It is the alt text, a description read aloud by a screen reader to someone who cannot see the image. fig.cap and fig.alt do different jobs: the caption is read by everyone and says what the figure shows, while the alt text describes what is in it for someone who cannot see it. Notice how long the one in the syllabus is; that is what a useful description costs. Every figure in your bioinformatics tutorial needs one.

✅ Checkpoint

The .png sits in Chapter_2/Figures/, next to your .Rmd rather than in Downloads or anywhere else on your computer.

Your document has a second-level Figures from files header (##) nested inside your first-level Figures section, and under it a labelled chunk calling knitr::include_graphics() with a path that starts from your own folder, not from the repository root.

In the knitted output the workflow diagram appears at full page width, captioned and numbered, and a sentence in your text refers to it by number as a link.

Move the .png out of Figures/ and knit again. You get an error rather than a silently missing figure, which is the behaviour you want: a broken path fails loudly. Put it back before moving on.

3.4.8 Cross-reference Tables and Figures in the Text (10 minutes)

Cross-referencing tables and figures in the main body of your R Markdown document can easily be done by using the \@ref() function implemented in the bookdown package (see Figure 3.10).

3.4.8.1 General Syntax

The general syntax is as follows:

# Cross-referencing tables in main body of text
Table \@ref(tab:code_chunk_ID)

# Cross-referencing figures in main body of text
Figure \@ref(fig:code_chunk_ID)

💡 More Info on the Syntax

  • It is worth mentioning that you need to manually add Table or Figure in front of \@ref(tab:code_chunk_ID) or \@ref(fig:code_chunk_ID). This design allows the user to choose how tables and figures are referenced in the text. Journals often have distinct formatting styles, which must be followed during submission.

3.4.8.2 Step-by-Step Procedure (~10 minutes)

  1. To cross-reference your table (labeled tabgrades) in the text, follow these steps:
    • Type the following sentence below your Tables header:
      The grading scale presented in the syllabus is available in Table \@ref(tab:tabgrades).
    • Knit your document using the Knit button in the editor toolbar (see Figure 3.11) to inspect the output and verify that the syntax and code are working correctly.
  2. To cross-reference your figure (labeled cars) in the text, follow these steps:
    • Type the following sentence below your Figures header:
      The plot of the cars data is available in Figure \@ref(fig:cars).
    • Knit your document using the Knit button in the editor toolbar (see Figure 3.11) to inspect the output and verify that the syntax and code are working correctly.
✅ Checkpoint

Below each of your two headers there is now a sentence containing \@ref(tab:tabgrades) or \@ref(fig:cars), with the word Table or Figure typed in front of it by hand.

In the knitted output those sentences read Table 1 and Figure 1, and both are links: click one and the page jumps to the table or the figure itself.

If you see \@ref(tab:tabgrades) printed literally, or a bold ??, the label inside the reference does not match the label on the code chunk. Compare the two character by character, and remember that tab: goes with tables and fig: with figures.

3.4.9 Cite References in the Text (15 minutes)

3.4.9.1 Introduction

In this section, we will cover the following topics before delving into the practical implementation of citing references in your R Markdown document:

We will then work through the procedures that implement citing in your document:

3.4.9.1.1 The Bibliography File

Pandoc can automatically generate citations in your text and a References section following a specific journal style (see Figure 3.10). To use this feature, you need to declare a bibliography file in the [YAML metadata section] under the bibliography: field.

In this course, we are working with bibliography files formatted using the BibTeX format. Other formats can also be used — please see this resource for more details.

Most journals allow citations to be downloaded in BibTeX format, but if this feature is not available, you can convert citation formats using online services (e.g., EndNote to BibTeX: https://www.bruot.org/ris2bib/).

3.4.9.1.2 The BibTeX Format

BibTeX is a reference management tool commonly used in conjunction with LaTeX (a Markup language) to format lists of references in scientific documents. It allows users to maintain a bibliographic database (.bib file) and cite references in a consistent and automated way.

Each entry in a .bib file follows this general structure:

Please find below an example of a reference formatted in BibTeX format:

@entrytype{citationIndentifier,
  field1 = {value1},
  field2 = {value2},
  ...
}

💡 Info on BibTeX format

  • @entrytype: Type of reference (e.g., @article, @book, @inproceedings).
  • citationIdentifier: Unique ID used to cite the entry in your R Markdown document.
  • fields: Key-value pairs with bibliographic information (e.g., author, title, year).

Here is an example associated with Baker (2016):

# Example of BibTex format for Baker (2016) published in Nature
@Article{Baker_2016,
  doi = {10.1038/533452a},
  url = {https://doi.org/10.1038/533452a},
  year = {2016},
  month = {may},
  publisher = {Springer Nature},
  volume = {533},
  number = {7604},
  pages = {452--454},
  author = {Monya Baker},
  title = {1,500 scientists lift the lid on reproducibility},
  journal = {Nature},
}
3.4.9.1.3 Citation Identifier

The unique citation identifier of a reference (Baker_2016 in the example above) is set by the user in the BibTeX citation file (see the first lines in the examples provided above). This identifier is used to refer to the publication in the R Markdown document and also enables citing references and generating the References section.

3.4.9.2 Steps to Complete Before Citing References in Your Text

  1. Save all your BibTeX-formatted references in a bibliography text file, and make sure to add the .bib or .bibtex extension.
  2. Place the bibliography file in your project folder (alongside your .Rmd file).
  3. Declare the name of your bibliography file in the [YAML metadata section] under the bibliography: field.
  4. References formatted in the BibTeX format are available in the following file:
    • Bibliography_Reproducible_Science_2.bib

3.4.9.3 Cite References in Your Text

We will examine the syntax for citing references, either as parenthetical citations or in-text citations.

  • Parenthetical Citation: The entire citation appears in parentheses, usually at the end of a sentence or clause.
  • In-Text Citation: The author’s name is part of the sentence, and the citation appears naturally in the flow of the text.
3.4.9.3.1 Parenthetical Citation

In this case, citations are placed inside square brackets ([]) (usually at the end of a sentence or clause) and separated by semicolons. Each citation must include a key composed of @ followed by the citation identifier (as stored in the BibTeX file).

Please find below some examples on citation syntax:

#Syntax
Blah blah [see @Baker_2016, pp. 33-35; also @Smith2016, ch. 1].
Blah blah [@Baker_2016; @Smith2016].

Once knitted (using the Knit button), the citation syntax is rendered as:

Blah blah (see Baker, 2016 pp. 33–35; also Smith et al., 2016, ch. 1).

Blah blah (Baker, 2016; Smith et al., 2016).

A minus sign (-) before the @ symbol will suppress the author’s name in the citation. This is useful when the author is already mentioned in the text:

#Syntax
Baker says blah blah [-@Baker_2016].

Once knitted, the citation is rendered as:

Baker says blah blah (2016).

3.4.9.3.2 In-Text Citation

In this case, in-text citations can be rendered with the following syntax:

#Syntax
@Baker_2016 says blah.
@Baker_2016 [p. 1] says blah.

Once knitted, the citation is rendered as:

Baker (2016) says blah.

Baker (2016 p. 1) says blah.

3.4.9.4 Step-by-Step Procedure (~10 minutes)

Now write both forms into your own document.

  1. If not done yet, open Chapter2_partC.Rmd in RStudio.
  2. Add a first-level header titled Citing references below your Figures section.
  3. Below that header, write a sentence citing Baker (2016) with parenthetical syntax:
Most researchers report having failed to reproduce another scientist's experiment [@Baker_2016].
  1. Write a second sentence citing the same reference with in-text syntax:
@Baker_2016 surveyed 1,576 researchers on reproducibility.
  1. Knit your document using the Knit button in the editor toolbar (see Figure 3.11) and check that both sentences render as citations rather than as raw text.

💡 Which form should you use?

Choose on grammar, not on preference. Use the parenthetical form when the source is evidence for the claim, and the in-text form when the authors are the subject of your sentence.

Notice that you typed @Baker_2016 both times. The difference comes entirely from the brackets, and the author name and year are pulled from the .bib file, so a misspelled author name cannot enter your manuscript by hand.

✅ Checkpoint

Bibliography_Reproducible_Science_2.bib is in your project folder, next to your .Rmd, and its name appears in the bibliography: field of your YAML metadata section.

Your document has a first-level Citing references header, and below it two sentences citing Baker (2016): one parenthetical ([@Baker_2016]) and one in-text (@Baker_2016 surveyed...).

In the knitted output the first renders inside brackets and the second as the author name followed by the year in brackets. A bibliography has also appeared at the very bottom of your document, listing Baker (2016), even though you have not yet added a References header. That header is the next section, and its job is to control where the bibliography lands.

If a citation renders as @Baker_2016 or as a bold ??, either the identifier does not match one in the .bib file, or the .bib file is not declared in your YAML.

3.4.9.5 Challenge (~5 minutes)

💡 Download a Citation in BibTeX Format

  • Visit this webpage.
  • Locate the Cite icon and download a citation in .bibtex format.
  • Open the .bibtex file in a text editor and inspect its contents.
  • What is the unique citation identifier of this article?
✅ Solution

The article is Hinsley et al. (2017), A review of the trade in orchids and its implications for conservation, published in the Botanical Journal of the Linnean Society.

The citation identifier is on the very first line, between @article{ and the first comma:

@article{THIS_IS_THE_IDENTIFIER,
    author = {Hinsley, Amy and de Boer, Hugo J and Fay, Michael F and ...},
    title = "{A review of the trade in orchids and its implications for conservation}",
    journal = {Botanical Journal of the Linnean Society},
    ...
}

The exact string you got depends on who exported it. Oxford Academic builds one automatically, and it will not look like the identifiers used in this course.

Now open Bibliography_Reproducible_Science_2.bib and search for botlinnean. This same article is already in your bibliography file, filed under Hinsley_et_al_2017, because I renamed it to something I could remember and type. That is the point of Citation Identifier: the identifier is set by whoever maintains the .bib file, not by the publisher.

You are free to rename an identifier, as long as you understand the consequence: every @key in every document that cites it must be changed to match, or those citations will silently render as ??.

3.4.10 Generate a References Section (10 minutes)

Upon knitting, a References section will automatically be generated and inserted at the end of your document (see Figure 3.10).

We recommend adding a level-1 “References” header immediately after the final paragraph of the document, as shown below:

last paragraph...

# References

The bibliography will be inserted after this header (please see the References section of this tutorial for more details).

3.4.10.1 Step-by-Step Procedure (~5 minutes)

  1. If not done yet, open Chapter2_partC.Rmd in RStudio.
  2. Add a level-1 References header after your last paragraph of content and before the # (APPENDIX) Appendices {-} header you built in PART B.
  3. Immediately below that header, add the line that pins the bibliography in place. Your document should then read:
@Baker_2016 surveyed 1,576 researchers on reproducibility.

# References

<div id="refs"></div>

# (APPENDIX) Appendices {-}

# Appendix 1
  1. Knit your document using the Knit button in the editor toolbar (see Figure 3.11) and check that the bibliography appears under References and above your appendices.

💡 Why the refs div matters

Without it, pandoc appends the bibliography to the very end of the document, which would push it below your appendices. With it, the bibliography is inserted exactly where the div sits, and the appendices stay last.

Either way you are not typing any references. The list is built from the citations pandoc found in your text, using the .bib file named in your YAML.

✅ Checkpoint

Your document has a first-level References header after its last paragraph of content, and before the # (APPENDIX) Appendices header you built in PART B.

In the knitted output the bibliography now sits under that header instead of being appended at the very end, and the appendices follow it.

The bibliography lists only the works you actually cited, not everything in the .bib file.

3.4.10.2 Challenge (~5 minutes)

💡 Where does the reference list actually come from?

  • Comment out one of your two citation sentences by putting <!– before it and –> after it. Knit, and look at the References section.
  • Now add a sentence citing a key that is not in your .bib file, such as [@NotARealKey]. Knit again, and look at both the body text and the References section.
  • What do these two experiments tell you about where the reference list comes from, and about what happens when you get a key wrong?
✅ Solution

Commenting out a citation removes its entry from the References section. The bibliography is not a list you maintain. It is rebuilt on every knit from the citation keys pandoc finds in your text, which means nothing you cited can be missing from it, and nothing you did not cite can appear in it. A reference list can therefore never drift out of step with the manuscript.

An unknown key renders as a bold ?? where the citation should be, R Markdown prints a warning in the console, and nothing is added to the References section. A mistyped key fails loudly instead of quietly producing a wrong citation, which is the behaviour you want.

Undo both experiments before moving on: restore the sentence you commented out, and delete the @NotARealKey sentence.

3.4.11 Format Citations to Journal Style (10 minutes)

In this section, we will explore how your bibliography can be automatically formatted to match a specific journal style. This is done by specifying a Citation Style Language (CSL) file in the YAML metadata section using the csl: field. The CSL file contains the formatting rules required to style both in-text citations and the bibliography according to the selected journal or publication.

3.4.11.1 What is the Citation Style Language?

The Citation Style Language (CSL) is an open-source project designed to simplify scholarly publishing by automating the formatting of citations and bibliographies. The project maintains a crowdsourced repository of over 8,000 free CSL citation styles. For more information, visit: https://citationstyles.org

3.4.11.2 CSL repositories

There are two main CSL repositories:

3.4.11.3 Step-by-Step Procedure (~10 minutes)

To learn this procedure and syntax, you will be tasked with downloading a .csl file and using it in your document (in place of AmJBot.csl).

Follow these steps to format your citations and bibliography according to a specific citation style (see Figure 3.10 for more details):

  1. Download a .csl file from the Zotero Style Repository.
    • To do this, type the name of your target journal in the search bar. Then, right-click on the corresponding style and select “Save Link As…” to download the .csl file. Save it in your project folder (see Figure 3.15).
      Procedure to save csl file for PNAS on the Zotero Style Repository.

      Figure 3.15: Procedure to save csl file for PNAS on the Zotero Style Repository.

  2. Make sure your .csl file is in your project folder (alongside your .Rmd file).
  3. Edit your YAML metadata section to specify the name of your .csl file.
  4. Knit your document using the Knit button. The Pandoc program will use the information specified in the YAML metadata section to format both the in-text citations and the bibliography section according to the citation style defined in the CSL file. Be sure to add a References header at the end of your .Rmd document.
✅ Checkpoint

A second .csl file, downloaded from the Zotero Style Repository, sits in your project folder alongside AmJBot.csl.

Your YAML csl: field names that new file, not AmJBot.csl.

In the knitted output, your citations and your bibliography look different from before: different punctuation, possibly a different author order, possibly numbers instead of names. Now switch the field back to AmJBot.csl and knit again. They change back.

Not one word of your text changed. You reformatted a whole manuscript to a second journal’s requirements by editing a single line, which is the argument for keeping content and formatting separate, made concrete.

3.4.12 All Set — Your Report Now Has Content

Chapter2_partC.Rmd is no longer an empty template. It holds a table and a figure that are rebuilt from code every time you knit, text that points at them by label rather than by number, and a bibliography that formats itself to whichever journal you name. Nothing in it has to be updated by hand when the data change or the target journal changes.

Compare your document against the finished template, Chapter_2_partC_template.Rmd. It contains the Tables and Figures sections with their labelled chunks, the cross-referencing sentences, the two citation styles, and the References header in its correct position ahead of the appendices. If something in your own file is missing or out of order, this is where to look.

💡 Info

  • Everything you built here (labelled chunks, cross-references, citations from a .bib file) is expected in your Bioinformatics Tutorial and your Individual Project.
  • PART D comes back to the two R code chunks at the top of the document and sets the global options that govern how all of this is knitted.

3.5 PART D: Advanced R and R Markdown Settings

3.5.1 Aim

By now you have written the same chunk options over and over: echo, warning, fig.cap, out.width. The aim of this tutorial is to set those options once, at the top of the document, so that every chunk inherits them and you only name an option on a chunk that needs to differ.

We will focus on learning how to:

  1. Create your PART D document and set your working directory, so that you can test code with the Run button before knitting (Create Your Document and Set Your Working Directory).
  2. Understand the options that govern every code chunk: how text output, code formatting, caching, plots and figure positioning are controlled (Set Global Options for Code Chunks).
  3. Assemble those options into a single setup chunk and apply it to your own document (Implement Global Options for Code Chunks).

3.5.2 Learning Outcomes

This tutorial provides students with the opportunity to learn how to:

We spend the first twenty minutes reviewing PART C before starting, because everything below is built on the document you produced there. That leaves enough of the class to build the setup chunk together; the plot chunk and the Challenge are homework, about eighteen minutes’ work.

Block Time Who
Review of PART C 20 min Instructor
Create Your Document and Set Your Working Directory 8 min Students
Set Global Options for Code Chunks 25 min Instructor
five-minute break 5 min
Implement Global Options: steps 1 to 5, the setup chunk 12 min Students
Subtotal in class 70 min
Implement Global Options: steps 6 to 9, the plot chunk 8 min Students, at home
Implement Global Options: Challenge 10 min Students, at home
At home 18 min

3.5.3 How This Tutorial Is Written

PART D follows the same protocol as PART B and PART C, with one difference worth knowing before you start: most of this tutorial is reference material.

Set Global Options for Code Chunks explains what each family of options does. There is nothing to type there and no Checkpoint, because nothing in your document changes yet. Read it the way you would read documentation, then come back to it later when you need to remember what cache or fig.pos does.

The two sections that do ask you to type (Create Your Document and Set Your Working Directory and Implement Global Options for Code Chunks) carry a Step-by-Step Procedure and a ✅ Checkpoint, and the last one closes with a 💡 Challenge and its ✅ Solution, exactly as in PART C. PART D does not finish in class: we build the setup chunk together, and you complete the plot chunk and the Challenge at home. That is what the homework for the following class refers to.

3.5.4 Create Your Document and Set Your Working Directory (8 minutes)

PART D starts from its own document. Make a copy of the file you finished in PART C rather than editing it: your PART C document is a deliverable in its own right, and you will want it intact when you come to compare the two.

💡 Approach to Create Your R Markdown

  • Create document: Make a copy of Chapter2_partC.Rmd in the same project folder and rename it Chapter2_partD.Rmd
  • Download document: If you had trouble with Chapter 2 - Part C, download the finished version, Chapter_2_partC_template.Rmd, and save it as Chapter2_partD.Rmd. You will also need Bibliography_Reproducible_Science_2.bib and AmJBot.csl in the same folder
  • Document title: Open the file in RStudio and edit the title in the YAML section to Chapter 2 – Part D
  • Test document: Knit it to confirm the copy works before you change anything

Now that you have a new file open, set the working directory to it. Unlike R scripts, where you must set your working directory or provide the path to your files, the approach implemented in an R Markdown document (.Rmd) automatically sets the working directory to the location of the .Rmd file when it knits. This behavior is managed by functions from the knitr package.

💡 More Info

  • knitr expects all referenced files to be located either in the same directory as the .Rmd file or in a subfolder within that directory.
  • This setup is designed to enhance the portability of your R Markdown project, which typically consists of multiple related files.

3.5.4.1 Step-by-Step Procedure (~8 minutes)

  1. Create Chapter2_partD.Rmd as described in the box above, and knit it once to confirm the copy works.
  2. With that file open and in focus, set the working directory to its location.

Before knitting your document, you will be testing your code (see Figure 3.13). To ensure smooth code testing, you need to set your working directory. In RStudio this is done by clicking (see Figure 3.16):

Session --> Set Working Directory --> To Source File Location

Screenshot of the RStudio menu bar with the Session menu open, the Set Working Directory submenu expanded and 'To Source File Location' highlighted, over an R Markdown editor pane in the background.

Figure 3.16: Snapshot of RStudio showing procedure to set your working directory to allow testing your code prior to knitting.

✅ Checkpoint

Chapter2_partD.Rmd exists, alongside Chapter2_partC.Rmd in Reproducible_Science/Chapters/Chapter_2, its YAML title reads Chapter 2 - Part D, and it knits. Your PART C file is untouched.

Your working directory is the folder holding that new .Rmd. Confirm it by typing getwd() in the Console: the path it prints should end in Chapter_2. This is the step worth doing deliberately, because you have just switched files, and the Console does not follow you when you do.

This matters only for testing code with the Run button. When you knit, knitr sets the working directory to the location of the .Rmd file itself, whatever the Console happens to be pointing at.

That difference is worth holding on to. Code that works in the Console but fails on knitting is very often code that reads a file from outside the document’s own folder, which is also the reason such a document will not run on anyone else’s machine.

3.5.5 Set Global Options for Code Chunks (25 minutes)

This section is reference material and I will walk you through it. There is nothing to type yet: you meet each family of options here, and apply all of them together in Implement Global Options for Code Chunks. The times below are how long we spend on each in class.

Topic Time What it controls
1 The opts_chunk$set() Function 5 min how a global option is declared
2 Text Output 4 min what the code prints into the document
3 Code Formatting 4 min how the code itself is laid out
4 Code Caching 4 min whether results are reused between knits
5 Plot Output 5 min figure format, resolution and location
6 Figure Positioning 3 min where a figure lands on the page
Total 25 min

3.5.5.1 Introduction

To improve code reproducibility and efficiency, and to comply with publication requirements, it is customary to include a code chunk at the beginning of your .Rmd file that sets global options for the entire document (see Figure 3.10). These settings relate to the following elements of your code:

These general settings will be configured using the opts_chunk$set() function provided by the knitr package (Xie, 2025b). The following website contains valuable information on code chunk options:

3.5.5.1.1 The opts_chunk$set() Function (~5 minutes)

opts_chunk$set() is used to modify the default global options in an .Rmd document. The name is worth unpacking, because opts_chunk is not a function, and that $ is not doing anything exotic.

opts_chunk is a list, and the things inside it are functions. You can check this yourself in the Console:

class(knitr::opts_chunk)  # 'list'
is.function(knitr::opts_chunk)  # FALSE
names(knitr::opts_chunk)  # 'get' 'set' 'delete' 'append' 'merge' 'restore' 'lock'

So opts_chunk$set(...) is the same $ you use to pull one element out of any named list, exactly as you will do in Create, Name, and Access Elements of a List in R in PART E. It reaches into the list, takes the element called set, and that element happens to be a function, so the () afterwards calls it. Nothing special is going on: opts_chunk has a set, it does not do anything by itself, which is why you cannot write opts_chunk(echo = TRUE).

💡 Why this is not a classical R function

The functions written so far in this course, and the ones you will write in PART E, are pure: you hand them arguments, they hand back a value, and nothing is remembered afterwards. Call square_number(2) a hundred times and you get 4 a hundred times.

opts_chunk is different. Its seven functions share one private environment that holds the current options, so they are not independent: 

knitr::opts_chunk$get("echo")      # TRUE
knitr::opts_chunk$set(echo = FALSE)
knitr::opts_chunk$get("echo")      # FALSE  <- get() sees what set() did

set() returns nothing useful; what it does is write the values into that shared environment.

And knitr reads from the same environment. That is the link that makes the whole thing work. When knitr processes your document it goes chunk by chunk, and for each one it asks opts_chunk what the current defaults are, then layers that chunk’s own options on top of them: 

knitr::opts_chunk$set(echo = FALSE, warning = FALSE)   # what your setup chunk does

knitr::opts_chunk$merge(list(echo = TRUE))             # what knitr does for a chunk
#   echo    -> TRUE     the chunk's own option wins
#   warning -> FALSE    the chunk said nothing, so your default applies

So the chain is: your setup chunk runs first and leaves values in the shared environment, and knitr then consults that same environment for every chunk and every piece of inline code that follows. A pure function could not do this, because it would forget the moment it returned, and your settings would reach nothing beyond the chunk that made them.

This pattern (a list of functions sharing a hidden environment) is a common way of building an object in R, and knitr uses it for opts_knit and knit_hooks as well.

Before starting, please note the following important points about the options:

  1. Chunk options must be written on a single line; no line breaks are allowed within chunk options.
  2. Avoid using spaces and periods (.) in chunk labels and directory names.
  3. All option values must be valid R expressions, just as function arguments are written.

We will discuss each part of the settings individually; however, these settings must be combined into a single code chunk in your document named setup (please see below for more details).

3.5.5.2 Text Output (~4 minutes)

This section covers settings related to the text output generated by code chunks.

Below is an example of options that can be applied across code chunks:

# Setup options for text results
opts_chunk$set(echo = TRUE, warning = TRUE, message = TRUE, include = TRUE)
3.5.5.2.1 Explanations of the Code
  • echo = TRUE: Include all R source code in the output file.
  • warning = TRUE: Preserve warnings (produced by warning()) in the output, as if running R code in a terminal.
  • message = TRUE: Preserve messages emitted by message() (similar to warnings).
  • include = TRUE: Include all chunk outputs in the final output document.

If you want some text results to use different options, please adjust those in their specific code chunks. This comment applies to all other general settings as well.

3.5.5.3 Code Formatting (~4 minutes)

This section covers settings related to code formatting (that is, how code is displayed in the final html or pdf document) generated by code chunks.

Below is an example of options that can be applied across code chunks:

# Setup options for code formatting
opts_chunk$set(tidy = TRUE, tidy.opts = list(blank = FALSE, width.cutoff = 60),
    highlight = TRUE)
3.5.5.3.1 Explanations of the Code
  • tidy = TRUE: Use formatR::tidy_source() to reformat the code. Please see the tidy.opts option below.
  • tidy.opts = list(blank = FALSE, width.cutoff = 60): A list of options passed to the function specified by the tidy option. Here, the code is formatted to avoid blank lines and has a width cutoff of 60 characters.
  • highlight = TRUE: Highlight the source code.

3.5.5.4 Code Caching (~4 minutes)

To compile your .Rmd document faster—especially when you have computationally intensive tasks—you can cache the output of your code chunks. This process saves the results of these chunks, allowing you to reuse the output later without re-running the code.

The knitr package provides options to evaluate cached chunks only when necessary, but this must be set by the user. This procedure creates a unique MD5 digest (a data storage technique) for each chunk to track changes. When the option cache = TRUE (there are other, more granular settings; see below) is set, the chunk will only be re-evaluated in the following scenarios:

  • There are no cached results (either this is the first time running or the cached results were moved or deleted).
  • The code chunk has been modified.

The following code allows you to implement this caching procedure in your document:

# Setup options for code caching
opts_chunk$set(cache = 2, cache.path = "cache/")
3.5.5.4.1 Explanations of the Code
  • In addition to TRUE and FALSE, the cache option also accepts numeric values (cache = 0, 1, 2, 3) for more granular control. 0 is equivalent to FALSE, and 3 is equivalent to TRUE.
    • With cache = 1, the results are loaded from the cache, so the code is not re-evaluated; however, everything else is still executed, including output hooks and saving recorded plots to files.
    • With cache = 2 (used here), the behavior is similar to 1, but recorded plots will not be re-saved to files if the plot files already exist—saving time when dealing with large plots.
  • cache.path = "cache/": Specifies the directory where cache files will be saved. You do not need to create the directory manually; knitr will create it automatically if it does not already exist.

3.5.5.5 Plot Output (~5 minutes)

Plots are a major component of your research and form the basis of your figures. You can take advantage of options provided by the knitr package to produce plots that meet publication requirements. This approach can save valuable time during the writing phase, as it eliminates the need to manually adjust figure size and resolution to comply with journal guidelines.

Below is an example of options that can be applied across code chunks:

# Setup options for plots The first dev is the master for
# the output document
opts_chunk$set(fig.path = "Figures_MS/", dev = c("png", "pdf"),
    dpi = 300)
3.5.5.5.1 Explanations of the Code
  • fig.path = "Figures_MS/": Specifies the directory where figures generated by the R Markdown document will be saved. As with caching, this folder does not need to exist beforehand; it will be created automatically. Files will be saved based on the code chunk label and assigned figure number.
  • dev = c("pdf", "png"): Saves each figure in both pdf and png formats.
  • dpi = 300: Sets the resolution (dots per inch) for bitmap devices. The resulting image dimensions follow the formula: DPI × inches = pixels. Check the submission guidelines of your target publication to ensure this value meets their requirements.

💡 More Than One Type of Figure

  • It is worth noting that you might use external figures in your .Rmd document. To avoid confusion between figures generated by the .Rmd file and those imported from outside sources, it is best practice to save them in two separate subfolders for more details).
3.5.5.5.2 Additional Plot Options

Some journals have specific requirements for figure dimensions. You can easily set these using the following option:

  • fig.dim: (NULL; numeric) — If a numeric vector of length 2, it specifies fig.width and fig.height. For example: fig.dim = c(5, 7).
    The unit for both dimensions is inches.

3.5.5.6 Figure Positioning (~3 minutes)

Positioning figures close to their corresponding code chunks is important for clarity and reproducibility. This can be controlled by adding another opts_chunk$set() option in your setup code chunk.

Use the fig.pos argument and set it to "H" to enforce figure placement near the relevant code.

## Locate figures as close as possible to requested
## position (=code)
opts_chunk$set(fig.pos = "H")

💡 Warning

  • This setting may cause errors when the .Rmd file is knitted to a pdf document. If this occurs, comment out the line of code using # and try knitting again.

3.5.6 Implement Global Options for Code Chunks (30 minutes)

Everything above described the options one family at a time. This section collects them into a single code chunk named setup, placed just below the YAML metadata section (see Figure 3.10 for its location), and then applies it to your own document.

Alongside the global settings, it is advisable to load the R packages your document needs in the same place (see Chapter 2 - Part B and Figure 3.10).

3.5.6.1 The setup Code Chunk

Below is the code for the setup chunk, built from the options presented above:

### Load packages Add any packages specific to your code
library("knitr")
library("bookdown")
### Chunk options: see https://yihui.org/knitr/options/ ###
### Text output
opts_chunk$set(echo = TRUE, warning = TRUE, message = TRUE, include = TRUE)
## Code formatting
opts_chunk$set(tidy = TRUE, tidy.opts = list(blank = FALSE, width.cutoff = 60),
    highlight = TRUE)
## Code caching
opts_chunk$set(cache = 2, cache.path = "cache/")
## Plot output The first dev is the master for the output
## document
opts_chunk$set(fig.path = "Figures_MS/", dev = c("png", "pdf"),
    dpi = 300)
## Figure positioning
opts_chunk$set(fig.pos = "H")
3.5.6.1.1 Settings of the setup R Code Chunk

When inserting the code above into an R code chunk (see Figure 3.10), please set the chunk options as follows:

  • setup: Unique ID of the code chunk.
  • include = FALSE: The code will be evaluated, and any plot files will be generated, but nothing will be written to the output document.
  • cache = FALSE: The code chunk will not be cached (see above for more details).
  • message = FALSE: Messages emitted by message() will not be included in the output.

3.5.6.2 The Plot Code Used Below

To see what the global settings actually do, you need a chunk that produces output. The code below generates the plot shown in Figure 3.17, and you will add it to your own document in the procedure that follows.

Plot of y ~ x.

Figure 3.17: Plot of y ~ x.

3.5.6.3 Step-by-Step Procedure (~20 minutes)

We do steps 1 to 5 together in class. Those five build the setup chunk, which is one complete piece of work: at the end of step 5 your document has its global options in place. Steps 6 to 9 are homework, and so is the Challenge that follows.

  1. Open Chapter2_partD.Rmd, the copy you made in Create Your Document and Set Your Working Directory.
  2. Check that your working directory still points at that file. Note: this matters if you want to test your R code before knitting.
  3. Insert an R code chunk (using the Insert button) directly below your packages code chunk, and name it setup. This chunk holds the global settings defined through opts_chunk$set().
  4. Copy the code from The setup Code Chunk into it.
  5. Set the chunk options as listed in Settings of the setup R Code Chunk.
  6. Homework starts here. Under your Figures header, insert another R code chunk and set its options as follows:
    • plotcache: Unique ID of the code chunk.
    • fig.cap = "Plot of y ~ x.": Figure caption.
    • fig.show = "asis": Figure display.
    • out.width = "100%": Figure width on the page.
  7. Copy the code from The Plot Code Used Below into that chunk, and test it with the Run button.
  8. Knit your document using the Knit button, and pay attention to what appears in your project folder, not only to the document.
  9. Cite your new figure in the text with the bookdown \@ref() function, as you did in PART C.
✅ Checkpoint

Directly below your packages chunk there is a chunk named setup, with the options include = FALSE, cache = FALSE, message = FALSE, containing the library() calls and the five opts_chunk$set() lines.

Under your Figures header there is a chunk named plotcache producing the scatter plot of y ~ x, captioned, and referenced from your text with \@ref(fig:plotcache).

Now look in your project folder. Two directories you never created have appeared: cache/, from cache.path, and Figures_MS/, from fig.path. Figures_MS/ holds both a .png and a .pdf of your plot, because dev = c('png', 'pdf') asked for two formats. Those image files are what you would send to a journal, produced as a side effect of knitting rather than exported by hand.

If Figures_MS/ is missing, your setup chunk did not run. Check that it is named exactly setup, and that include = FALSE has not been mistyped as eval = FALSE: the first hides the output, the second stops the code running at all.

3.5.6.4 Challenge (~10 minutes)

💡 Which setting wins?

  • Your setup chunk sets echo = TRUE for every chunk in the document. Change it to echo = FALSE, knit, and look at what happened to the R code in your output.
  • Now add echo = TRUE to the options of your plotcache chunk only, and knit again.
  • Which of the two settings won for that chunk, and what does that tell you about what opts_chunk$set() actually does?
  • Put both back the way they were before moving on.
✅ Solution

The chunk-level option wins. opts_chunk$set() does not force anything on your chunks: it sets the default for the document. Any option written in an individual chunk header overrides that default, for that chunk alone.

That is why everything you wrote in PART C still behaves exactly as you wrote it. The echo = FALSE on your tabgrades chunk keeps working whatever the global default becomes, because a chunk option always beats a global one.

This is what makes global options worth setting at all. You choose the behaviour you want almost everywhere, declare it once at the top, and then name an option on a chunk only where that chunk genuinely needs to differ. In a document where every chunk repeats the same six options, changing your mind means editing every chunk, and that is exactly the kind of manual, repeated edit this course exists to remove.

3.6 PART E: User-Defined Functions in R

3.6.1 Aim

The aim of this tutorial is to provide an introduction to functions; more specifically, we will be studying user-defined functions (UDFs) as implemented in R.

💡 UDFs 101

  • What are UDFs? UDFs allow users to write their own functions and make them available in the R Global Environment.
  • How do they work? UDFs are saved in .R scripts and are made available to users using the source() function, which reads the file content and loads the UDFs.
  • Your UDF collection can ultimately lead to creating R packages.

3.6.2 Learning Outcomes

By the end of this tutorial, students will be able to:

PART E runs over three classes, and we write the code together as we go. The times below are a pace rather than a deadline, and they have been revised to match how the sessions actually ran: live coding with a room full of questions takes longer than the nominal figures, which is as it should be.

Class 1 opens with a 20-minute challenge from PART C, filling a gap we skipped: inserting a figure that already exists as an image file. Class 2 opens by finishing class 1’s exercise together, and class 3 opens with the whiteboard presentations from class 2. Only class 1 leaves anything for home, marked in the table below; classes 2 and 3 are finished in the room.

Class 1 — what a function is, and your first UDF

Block Time Who
👥 Insert a Figure from an Image File (PART C) 20 min Students
Define a Function and Understand its Applicability 6 min Instructor
Syntax to Implement UDFs in R 9 min Instructor
five-minute break 5 min
Apply Best Practices for Writing Code 7 min Instructor
Load and Call UDFs 8 min Instructor
UDFs That Return a Single Value: The Task 15 min Instructor, live coding
Total in class 70 min
UDFs That Return a Single Value: implement it and apply it 15 min Students, at home

Class 2 — lists, and returning more than one value

Block Time Who
UDFs That Return a Single Value: implement it and apply it 20 min Students, finishing class 1 together
Create, Name, and Access Elements of a List in R 20 min Instructor live codes, students follow along
five-minute break 5 min
UDFs that Return Multiple Values: pairs plan and build 25 min Students, in pairs
Total in class 70 min

Class 3 — showing your work, and failing well

Block Time Who
UDFs that Return Multiple Values: pairs present at the whiteboard 10 min Students, in pairs
Use Logical Operators within R Functions 3 min Instructor
Implement Defensive Programming to Improve Code Robustness 25 min Instructor live codes, students follow along
Total in class 38 min
Use Your UDFs in an R Markdown Document 8 min Students, at home

3.6.3 How This Tutorial Is Written

PART E is taught differently from PART C and PART D. We write the code together, live, in class. I type, you type alongside me, and we stop whenever something does not run.

That is why the exercises here are built as a task, then pseudocode, then code, in that order. Pseudocode first is not ceremony: deciding what a function must do, in plain sentences, before writing any R is the habit this part exists to teach, and it is the same habit the bioinformatics tutorial will ask of you.

Each exercise closes with a ✅ Checkpoint, and those matter more here than anywhere else in this chapter. Live coding only works if nobody is quietly stuck, so a Checkpoint is where we all stop and confirm the same thing runs on every machine before the next piece is built on top of it. If yours does not, say so at that point rather than afterwards.

💡 Disclaimer: this part is not written in R Markdown

  • You will not create an .Rmd file in PART E. Everything here is written in .R scripts: plain R code, with no YAML, no chunks and nothing to knit.
  • Your functions go in a new folder called R_functions, at the root of your Chapter_2 folder, with one function per script. That convention is worth adopting now rather than later: a function kept in its own file can be loaded by any document that needs it, instead of being copied into each one and drifting out of step.
  • You will also create one working script, Chapter2_partE.R, directly in Chapter_2. That is where you load your functions and try them out.
  • R Markdown returns at the very end, in Use Your UDFs in an R Markdown Document, where these scripts get connected to the document you have been building since PART B. Until then, forget about knitting.
  • To reach the learning outcomes you will complete three guided exercises. They use mathematical examples deliberately, so that nothing depends on knowing a particular dataset. Before the exercises I will give context on what functions are, when to use them, and best practice for pseudocode and R code (taken further in Chapter 5).

By the end of PART E your Chapter_2 folder will look like this: 

Reproducible_Science/Chapters/Chapter_2/
|-- Chapter2_partC.Rmd        <- from PART C
|-- Chapter2_partD.Rmd        <- from PART D
|-- Chapter2_partE.R          <- your working script for PART E
`-- R_functions/              <- NEW: one file per function
    |-- square_number.R
    |-- my_log_square.R
    `-- exp_number.R

3.6.4 Define a Function and Understand its Applicability (8 minutes)

3.6.4.1 What is a Function?

In programming, you use functions to incorporate sets of instructions that you want to use repeatedly or that, because of their complexity, are better self-contained in a subprogram and called when needed.

A function is a piece of code written to carry out a specified task; it may or may not accept arguments or parameters, and it may or may not return one or more values.

3.6.4.2 Functions in R

There are many terms used to define and describe functions—subroutines, procedures, methods, etc.—but for the purposes of this tutorial, you will ignore these distinctions, which are often semantic and reminiscent of older programming languages (see here for more details on semantics). In our context, these definitions are less important because, in R, we only have functions.

3.6.4.3 R Syntax for Functions

In R, you define a function with the following construct:

function(arguments){
   body
}

A function has two parts, and both come up constantly in PART E:

  • arguments are the inputs: the values you hand the function when you call it. A function can take none, one, or several.
  • body is the code between the curly braces: what the function does with those inputs.

When you use a built-in function, the only two things you have to get right are which arguments to give it and what it gives back. To see which arguments a function expects, open its documentation from the R console:

#General syntax
?function() 

#Example with read.csv()
?read.csv()

A note if you look this up yourself. R’s own help page for function writes the same construct as function(arglist){ body }. arglist and arguments mean exactly the same thing; this course says arguments throughout, because that is the word R uses in its error messages and in every other help page you will read.

3.6.5 Syntax to Implement UDFs in R (10 minutes)

R allows users to define their own functions (UDFs), which are based on the following syntax:

function.name <- function(arguments){
   computations on the arguments
   some more code
   return value(s)
}

💡 In most cases, an R function has the following components:

  • A name: In our example, function.name.
  • Arguments (arguments): These are inputs to the function, declared within the parentheses () following the keyword function.
  • A body: The body is the code enclosed within curly braces {}, where the computation is carried out.
  • One or more return values: These values are output by the user-defined function (UDF).
  • Definition syntax: Overall, you define the function similarly to variables, by “assigning” the directive function(arguments) to the “variable” function.name, followed by the function body.

3.6.6 Apply Best Practices for Writing Code (8 minutes)

This topic will be explored in more detail in Chapter 5, but here we introduce some best practices for writing clear and maintainable code.

A key part of this process involves starting with pseudocode before transitioning to actual R code. Pseudocode provides a high-level, language-agnostic description of the tasks your function needs to perform. It allows you to plan the structure and logic of your function before dealing with R syntax.

Once the pseudocode is defined, the next step is to translate each task into actual R code. This involves identifying existing R functions that can perform the required operations. If suitable functions do not exist, you may need to write your own—potentially supported by additional pseudocode to clarify the logic.

Below, you will find more detailed definitions of the two key concepts introduced here: pseudocode and code writing/implementation.

3.6.6.1 Writing Pseudocode

Pseudocode is an informal, high-level description of the operating logic of a computer program or algorithm. It uses the structural conventions of a programming language (in this case, R) but is intended for human understanding rather than machine execution.

In this stage, you outline the major steps of your function and break them down into associated tasks. You then link these steps to specific R functions—either existing ones or those you will need to develop. This provides the backbone of your code and supports the transition to implementation.

3.6.6.2 Writing Code

Writing clear, reproducible code has (at least) three main benefits:

  1. It makes it much easier to revisit your code months later—whether you’re returning to an old project or making revisions after peer review.
  2. It allows others to more easily scrutinize the results of your analysis, helping to establish their validity.
  3. Clean, well-documented, and reproducible code can encourage others to adopt any new methods you’ve developed (and can lead to more citations).

3.6.7 Load and Call UDFs (9 minutes)

When working on your project, it is highly likely that you will develop multiple UDFs tailored to your research. In this case, it is recommended to create a dedicated folder named R_functions within your project directory (see Figure 3.18). Save each UDF as a separate file (e.g., check.install.pkg.R) inside the R_functions folder.

Screenshot of a macOS Finder window for a folder named Project_ID, listing six subfolders (Data, MS, Figures and Tables, Output, Reports, R_functions) above five numbered R scripts (01_download_data.R through 05_generate_figures.R) and a README.md file.

Figure 3.18: Example file structure of a simple analysis project. See Chapter 4 for more details

3.6.7.1 The source() Function

Once your project is properly structured (see Figure 3.18), it becomes easy to load specific UDFs stored in separate R scripts using the source() function.

For example, to load the check.install.pkg() function (saved as R_functions/check.install.pkg.R) into the Global Environment, enter the following code in the R console (see Figure 3.19):

source("R_functions/check.install.pkg.R")
Screenshot of the full RStudio window: the source editor on the left shows the check.install.pkg function definition, the console below shows source() and View() calls being run, and the Environment panel on the right lists check.install.pkg under a Functions heading.

Figure 3.19: Snapshot of RStudio showing output of source() function and the UDF made available in the Global Environment. You can also view the code unperpinning the UDF.

Below is an example of R code that loads all your UDFs stored in the R_functions folder. This approach is especially useful when you have multiple UDFs associated with your analysis pipeline.

### Load all UDFs stored in R_functions
# 1. Create vector with names of UDF files in R_functions
# (with full path)
files_source <- list.files("R_functions", full.names = T)
# 2. Iterative sourcing of all UDFs
sapply(files_source, source)

3.6.7.2 Call Your UDFs

Once your UDFs are loaded into the Global Environment (see Figure 3.19 for an example), you can call them by typing their names directly into the R console. For example:

# Check if knitr package is installed, if not install it
# and then load it
check.install.pkg(pkg = c("knitr"))
## [1] "Check if packages are installed"
## [1] "Load packages"
## knitr 
##  TRUE

3.6.8 Develop, Implement, and Apply UDFs That Return a Single Value (30 minutes)

In this section, we aim at developing UDFs that return a single value. But how do we tell a function what to return?

In R, this is done using the return() function.

To learn this skill, we will work on a challenge.

3.6.8.1 The Task (~15 minutes)

We are tasked with

Developing, implementing, and applying a UDF to calculate the square of a number.

3.6.8.1.1 Writing Pseudocode

Let’s begin by understanding this challenge from a mathematical perspective. Then, we will develop a pseudocode to solve it computationally. Finally, we will implement our solution in R.

In mathematics, the squared symbol (\(^2\)) is an arithmetic operator that signifies multiplying a number by itself. The “square” of a number is the product of the number and itself. Multiplying a number by itself is called “squaring” the number.

Although this task is straightforward, the pseudocode to implement the function requires the following steps:

  1. Store the number inputted by the user into an argument called base, as part of the function’s argument(s).
  2. Calculate the square of base by multiplying the number by itself.
  3. Save the result of step 2 into an object called sq.
  4. Return the sq object to the user (a single numeric value). To do this, we will use the return() function.

💡 Let’s talk about the class of input/output data

  • Input: The class of the input data must be numeric.
  • Output: The class of the output data must also be numeric.
  • Tip: You can check the class of an object using the class() function.
  • The class() function is also useful when implementing defensive programming.
3.6.8.1.2 Writing Code

In this section, we will implement the pseudocode outlined above in a function called square_number().

This function takes one argument from the user (base, a number) and returns the square of that number (i.e., base * base).

## Create a UDF in R to calculate square number:
# - argument: base (= one number)
# - output: square of base (= one number)
square_number <- function(base){
  #Infer square of base and save it into object
  sq <- base*base
  
  #Return sq object
  return(sq)
}

3.6.8.2 Implement and Load the UDF in R (~10 minutes)

Follow these steps to implement and load the UDF in R:

  1. Create a new folder named R_functions inside your Chapter_2 folder.
  2. In RStudio, create a new .R script and save it as square_number.R in Chapter_2/R_functions.
  3. Copy the code for the square_number function into your new R script.
  4. Load the function into the Global Environment by executing all lines related to square_number.
  5. Verify that square_number has been loaded by checking the Environment panel in RStudio (see Figure 3.20).
Close-up screenshot of the RStudio Environment tab set to Global Environment, showing a single Functions section with one entry, square_number, described as function (base).

Figure 3.20: Close-up of the Environment panel in RStudio showing that the UDF is loaded in the Global Environment and can be used.

💡 Use the source() function to load the UDF.

  • You can also load the square_number() function by typing the following command in the console: source(“R_functions/square_number.R”).

3.6.8.3 Apply Your Function (~5 minutes)

The R language is quite flexible and allows functions to be applied to a single value (e.g., base = 2) or a vector (e.g., base = c(2, 4, 16, 23, 45)). Let’s further explore this concept by completing the following tasks in your R script:

  1. Create a new .R script titled Chapter2_partE.R saved in Chapter_2/.
  2. Set your working directory to the location of that script: Session --> Set Working Directory --> To Source File Location, following the procedure in Create Your Document and Set Your Working Directory. This step is not optional here. An .Rmd has knitr to set the working directory for it; a plain .R script has nothing of the kind, so the relative path in the next step only resolves if R is actually pointed at Chapter_2.
  3. Edit your .R script as follows to load your UDF using the source() function.
### Source UDFs Source the square_number function
source("R_functions/square_number.R")
  1. Apply your UDF to a single value:
### Apply your UDFs Square number of 2
square_number(base = 2)
## [1] 4
  1. Apply your UDF to a vector containing multiple values:
# Create vector with numbers
bases <- c(2, 4, 16, 23, 45)
# Apply function to vector
square_number(base = bases)
## [1]    4   16  256  529 2025
✅ Checkpoint

Chapter_2/R_functions/square_number.R exists and contains only the square_number function.

square_number appears under Functions in your RStudio Environment panel. If it does not, the file was saved but never run: source("R_functions/square_number.R") loads it.

Chapter_2/Chapter2_partE.R exists, begins with that source() call, and running it end to end prints 4 for square_number(base = 2) and five values for the vector.

If source() fails with “cannot open file”, your working directory is not Chapter_2. Check it with getwd() and set it again. This is the single most common way this exercise goes wrong, and it has nothing to do with your function.

Notice what the vector case tells you. You wrote base*base for a single number, and it worked unchanged on five numbers at once. That is R being vectorised, and it is the reason a UDF written for one value usually scales to a whole dataset without a loop.

The first two points are what we do together in class; the third is your homework. If any of them is not true on your machine when we next meet, say so at the start rather than at the end: everything in class 2 is built on top of this function.

3.6.9 Create, Name, and Access Elements of a List in R (12 minutes)

In this section, we introduce lists in R and learn how to create, name, and access list elements. We will also explore how lists can contain mixed data types and nested structures.

The procedures and syntax covered here will be applied in the next section, where we investigate how to implement user-defined functions (UDFs) that return multiple values.

More specifically, this section provides procedures and syntax for:

💡 Create Your R Script to Practice

  • R Script: If not done yet, create an R script titled Chapter2_partE.R and save it in your project folder (Chapter_2).
  • Copy and paste the code provided in this section into your script to practice and reinforce these new protocols and syntax.

3.6.9.1 What is a List in R?

Lists are R objects that can contain elements of different types, such as numbers, strings, vectors, or even another list. A list can also include a matrix or a function as an element.

Lists are created using the list() function.

3.6.9.2 How to Create a List

Below is an example of how to create a list containing strings, numbers, vectors, and logical values using the list() function:

# Create a list containing strings, numbers, vectors and a
# logical value
list_data <- list("Red", 51.3, 72, c(21, 32, 11), TRUE)
# Print object
print(list_data)
## [[1]]
## [1] "Red"
## 
## [[2]]
## [1] 51.3
## 
## [[3]]
## [1] 72
## 
## [[4]]
## [1] 21 32 11
## 
## [[5]]
## [1] TRUE

3.6.9.3 How to Name List Elements

List elements can be assigned names using the names() function, allowing them to be accessed by those names, as shown below:

# Create a list containing a vector, a matrix and a list
list_data <- list(c("Jan", "Feb", "Mar"), matrix(c(3, 9, 5, 1,
    -2, 8), nrow = 2), list("green", 12.3))
# Give names to the elements in the list
names(list_data) <- c("1st_Quarter", "A_Matrix", "An_inner_list")
# Show the list
print(list_data)
## $`1st_Quarter`
## [1] "Jan" "Feb" "Mar"
## 
## $A_Matrix
##      [,1] [,2] [,3]
## [1,]    3    5   -2
## [2,]    9    1    8
## 
## $An_inner_list
## $An_inner_list[[1]]
## [1] "green"
## 
## $An_inner_list[[2]]
## [1] 12.3

3.6.9.4 How to Access List Elements

Elements of a list can be accessed using their index or, in the case of named lists, using their names (combined with the $ symbol).

You have met this $ already. In PART D you used opts_chunk$set() to set the global options for your code chunks, and that $ is doing exactly what it does here: reaching into a named list and pulling out one element. The only difference is what the element turns out to be. In opts_chunk it was a function, so a () followed it to call it; here it is a matrix, so there is nothing to call.

The following example illustrates how to access list elements using both methods:

# Create a list containing a vector, a matrix and a list
list_data <- list(c("Jan", "Feb", "Mar"), matrix(c(3, 9, 5, 1,
    -2, 8), nrow = 2), list("green", 12.3))
# Give names to the elements in the list
names(list_data) <- c("1st_Quarter", "A_Matrix", "An_inner_list")
# Access the first element of the list
print(list_data[[1]])
## [1] "Jan" "Feb" "Mar"
# Access the thrid element. As it is also a list, all its
# elements will be printed
print(list_data[[3]])
## [[1]]
## [1] "green"
## 
## [[2]]
## [1] 12.3
# Access the list element using the name of the element
# (combined with $ in front)
print(list_data$A_Matrix)
##      [,1] [,2] [,3]
## [1,]    3    5   -2
## [2,]    9    1    8

This is why naming list elements matters. list_data$A_Matrix says what you are asking for, while list_data[[2]] only says where it happens to sit, and will quietly return the wrong thing if the order ever changes.

One more thing before you move on. Naming took two steps here: build the list, then add the names with names(). In the next learning outcome you will meet the shorter form, which creates a list and names its elements in a single call. That is the form a UDF uses to hand back several values at once, so you will meet it there rather than here.

We will now apply what you have learned to return multiple values from a UDF.

3.6.10 Develop, Implement, and Apply UDFs that Return Multiple Values (25 minutes)

In this section, we will focus on developing UDFs that return multiple values.

To learn this skill, we will work on a challenge.

3.6.10.1 The Task

We are tasked with

Developing, implementing, and applying a UDF to calculate both the logarithm and the square of a number.

To accomplish this, we will collect the different outputs into a list, which will then be returned to the user using return().

3.6.10.2 Student Activity (~25 minutes)

Work in pairs. Two people is the right size for this: both of you can see the screen, neither of you can sit it out, and you have to say your reasoning out loud to somebody before it becomes code. If your partner is away today, join another pair rather than working alone.

This exercise straddles two classes. You plan and build your function in the session where it is introduced, and the pairs present at the start of the following class, one after the other, in about ten minutes altogether. That gap is useful rather than accidental: it gives you time to get the function actually working, and the presentations are not optional extras, they are where the exercise pays off.

  1. Plan it on paper first (~6 minutes). Nobody opens RStudio yet. Agree three things and write them down:
    • Input and output. What does the function take, and what exactly does it hand back? Give the class of each. “Two numbers” is not an answer; “one numeric value in, a list of two named numeric values out” is.
    • The steps. In plain sentences, one per line and in the order they happen, what has to occur between the input arriving and the output leaving.
    • The names. Decide now what you will call the argument, each intermediate object, and each element of the list you return. Naming is not tidying up afterwards: a name that says what the thing is turns the next step into transcription, and tells whoever reads the function later what you meant by it.
  2. Implement and test it (~9 minutes). Turn the plan into a UDF, saved as my_log_square.R in Chapter_2/R_functions, then source() it and apply it to a number.
  3. Present at the whiteboard (10 minutes, 2 to 3 per pair). Each pair takes the board in turn and shows the class both halves: the plan you wrote, and the code it became. With four pairs there is no time to walk through everything, so pick the one place where the plan and the code differed and talk about that: the decision you had to make while typing that the plan had left open. That gap is the most useful thing in this exercise, because it is exactly where your bioinformatics tutorial will lose a reader who cannot see what you were thinking.
✅ Solution

Please find below the solution proposed by the instructor:

##Create a user defined function in R to calculate log and square of a number:
# argument: base (= one number)
# output: log and square of base (= two numbers) returned in a list
my_log_square <- function(base){
  ## 1. Infer log and square of base
  #log (base 10)
  log_value <- log(base)
  #Square of base
  square_value <- base^2
  
  ## 2. Return both objects/values in a list
  # Name objects in the list to help user determine what value refers to which output
  return(list(log_val = log_value, square_val = square_value))
}

# Call the function
my_log_square(base = 2)
## $log_val
## [1] 0.6931472
## 
## $square_val
## [1] 4

The list is what makes this work. A function returns exactly one object, so returning two numbers means putting both inside a single container first. Naming the list elements (log_val, square_val) is not decoration: without names the caller has to remember that the first element is the logarithm, and that is exactly the kind of undocumented assumption that makes an analysis hard to reproduce.

✅ Checkpoint

Chapter_2/R_functions/my_log_square.R exists, and source()-ing it puts my_log_square in your Environment panel.

Calling my_log_square(base = 2) prints a list of two named elements, not two loose numbers.

You can pull one value out on its own: my_log_square(base = 2)$square_val returns 4. Try it. That is why the names matter, and it is how you will use a multi-value UDF inside a larger script.

3.6.11 Use Logical Operators within R Functions (3 minutes)

The next section checks two things at once (is this argument a number, or is that one not?), which needs a logical operator. We meet them here first so that the check reads as English when you get there.

In a nutshell, the most commonly used logical operators in R are:

  • AND (&): Returns TRUE only if both values are TRUE.
  • OR (|): Returns TRUE if at least one of the values is TRUE.
  • NOT (!): Negates the logical value it is applied to.

You can learn more about logical operators (including practice exercises) on this website.

3.6.12 Implement Defensive Programming to Improve Code Robustness (25 minutes)

3.6.12.1 Introduction

Defensive programming is a technique used to ensure that code fails with well-defined errors—that is, in situations where you know it should not work. The key idea is to ‘fail fast’ by ensuring the code throws an error as soon as something unexpected occurs. This may require a bit more effort from the programmer, but it makes debugging much easier later on.

3.6.12.2 Case Study (~22 minutes)

We are tasked with

Developing, implementing, and applying a UDF that raises a number to a power, and stops with a clear message when the inputs are not numbers.

The function itself is a power function, y = x^n, taking two arguments. The arithmetic is deliberately trivial: what this exercise is really about is the guard you put in front of it, and what that guard says when somebody uses the function wrongly.

💡 You work this section in your own script, not on this page

  • Create the script now: an R script titled exp_number.R, saved in Chapter_2/R_functions, holding the function below and nothing else.
  • Everything that follows happens in that file. You will source() it to load the function, call it with input that breaks it, then edit the file to add the defensive check and source() it again to see the difference. Reading the three code blocks below without running them teaches you almost nothing: the whole point is watching a useless error message become a useful one on your own machine.
  • Keep your working script to hand too. Load and call exp_number() from Chapter2_partE.R, as you did for the other two functions, so your calls are saved rather than typed into the console and lost.
  • Expect to see errors, and do not be alarmed by them. Two of the examples below are supposed to fail. That is what is being taught here: code that fails clearly and early is better than code that fails vaguely and late. 
# Define a power function (exp_number): y = x^n
# - Arguments: base (= x) and power (= n)
# - Output: a number (y)
exp_number <- function(base, power){
  #Infer exp (y) based on base (x) and power (n) (y=base^power)
  exp <- base^power
  
  #Return exp object
  return(exp)
}

# Call function
exp_number(base = 2, power = 5)
## [1] 32
3.6.12.2.1 Let’s Test the UDF!

First, load the function. You saved exp_number in a script of its own, so it does not exist in your Global Environment until you source() that file, exactly as in Load and Call UDFs:

### Source the UDF
source("R_functions/exp_number.R")
### Check it works on valid input before trying to break it
exp_number(base = 2, power = 5)

💡 Re-source the file after every edit

source() puts a copy of the function into your Global Environment. Editing and saving the script does not change that copy: R keeps using the one it already has until you source() the file again.

You are about to edit this very function in the next section, so you will need this. If a change you just made appears to have no effect, this is almost always why, and it is one of the most common ways an afternoon disappears.

You can now apply defensive programming to the exp_number function defined above. The function requires that both arguments be of class numeric. If you provide a string (e.g., a word) as input, the function will return an error:

# Example where we don't respect the class associated with
# the argument base
exp_number(base = "hello", power = 5)
## Error in base^power: non-numeric argument to binary operator
# And the same with a '2', which LOOKS like a number but is
# not one: the quotes make it a character string, so the
# arithmetic still fails
exp_number(base = "2", power = 5)
## Error in base^power: non-numeric argument to binary operator
3.6.12.2.2 Let’s Implement Defensive Programming in the UDF!

The fix is to check the inputs before doing any arithmetic with them, and to stop with a message of our own if they are not what the function needs. That check is built from three pieces:

  1. is.numeric() asks of one object: is this a number? It returns a single TRUE or FALSE, and the ! in front of it flips the question to is this not a number?
  2. A logical operator combines the two answers, because there are two arguments to check. This is exactly what Use Logical Operators within R Functions was for: we use OR (|), so the test is true when at least one of the arguments is wrong.
  3. stop() halts the function immediately and prints the message you give it, instead of letting R struggle on and fail later in its own words. call. = FALSE is worth adding: without it R prefixes your message with the whole failing call, as in Error in exp_number(base = "hello", power = 5): ..., which repeats what the user just typed. With it, they see Error: Both base and power inputs must be numeric and nothing else.

💡 Why | and not &

This is the easiest place to get a guard backwards. With AND (&) the function would only complain when both arguments were wrong, so exp_number(base = “hello”, power = 5) would walk straight past the check and fail later with R’s cryptic non-numeric argument to binary operator.

OR (|) is what you want: one bad input is enough to refuse, and one bad input is by far the commonest case. A guard that only fires when everything is wrong is barely a guard at all.

This routine can be implemented as follows:

# Define a power function (exp_number): y = x ^n
# - Arguments: base (= x) and power (= n)
# - Output: a number (y)
exp_number <- function(base, power){
    # This if statement tests whether base and power are numbers.
    # If either is not, it stops and returns a meaningful message.
    if(!is.numeric(base) | !is.numeric(power)){
          # call. = FALSE drops R's "Error in exp_number(...)" prefix, so the
          # user sees only the message we wrote
          stop("Both base and power inputs must be numeric", call. = FALSE)
    }
    #If classes are good then infer exp
    exp <- base^power
    
    # Return exp object
    return(exp)
}

# Call function
exp_number(base = "hello", power = 5)
## Error: Both base and power inputs must be numeric
# The guard catches a quoted "2" as well, which is the case that surprises people
exp_number(base = "2", power = 5)
## Error: Both base and power inputs must be numeric

💡 “2” is not 2

This is the example worth pausing on. 2 is a number; “2” is a character string that happens to contain a digit, and the quotation marks are the whole difference: 

is.numeric(2)     # TRUE
is.numeric("2")   # FALSE
class("2")        # "character"

R will not quietly convert one to the other, so “2”^5 fails just as “hello”^5 does. Our guard is right to refuse it: a function that silently accepted “2” would be guessing at what the user meant, and the next quoted value might be “two” or “2,5”.

If you genuinely have numbers arriving as text, which happens constantly when reading data files, convert them deliberately with as.numeric() before calling the function. That way the conversion is a decision you made and can see, not something the function did behind your back.

💡 Why is.numeric() and not class()?

You may see this check written as class(base) != “numeric”. It looks equivalent and is not, in two ways that matter:

  • class(2L) is “integer”, not “numeric”, so that version rejects whole numbers typed as 2L, which are perfectly valid input.
  • class() can return more than one value (a matrix gives c(“matrix”, “array”)), and comparing a length-2 vector inside if() is an error in current R: the condition has length > 1.

So the guard meant to produce a clear message would itself crash with an obscure one. is.numeric() always returns a single TRUE or FALSE and accepts both integers and doubles. Defensive code has to be more robust than the code it protects, not less.

Although debugging this particular error would not take long, in more complex functions you are more likely to meet vague error messages, or code that runs for some time before failing. By adding checks like this one you detect unexpected behaviour earlier and get a message that tells you what to fix.

3.6.13 Use Your UDFs in an R Markdown Document (8 minutes)

Everything so far has lived in .R scripts. This last section connects them back to the R Markdown document you have been building since PART B, which is where your UDFs will actually be used.

The link is the same source() function you have been using in the console. Because knitr sets the working directory to the location of the .Rmd file (see Create Your Document and Set Your Working Directory), a relative path is all you need.

3.6.13.1 Step-by-Step Procedure

  1. Copy your R_functions folder into the folder holding Chapter2_partC.Rmd, if it is not already there.
  2. Open Chapter2_partC.Rmd and find the setup chunk you built in PART D.
  3. Add a source() call for each UDF, below the library() calls:
### Source UDFs
source("R_functions/square_number.R")
source("R_functions/my_log_square.R")
  1. Anywhere below that chunk, call one of them from a normal code chunk, exactly as you would in the console:
square_number(base = 2)
  1. Knit the document.

💡 Sourcing a whole folder

Listing every file by hand does not scale. Once you have more than a handful of UDFs, source the folder instead, which picks up any script you add later without further edits: 

ToSource <- list.files(path = "R_functions", pattern = "\\.R$", full.names = TRUE)
sapply(ToSource, source)

That pattern is a regular expression, not a filename wildcard. Before reading it, one thing that catches everybody: the code above has two backslashes, and the pattern itself has one. That is not a typo. There are two layers here:

  • What the regular expression needs is \. : a backslash followed by a dot, meaning a literal dot.
  • What you type in R is "\\.R$", with two of them, because a backslash is also R’s own escape character inside a quoted string. R consumes the first backslash and stores the second.

You can watch that happen in the console:

# You type two backslashes, but the string only holds one
nchar("\\.R$")
## [1] 4
# Print it and you see what the regex engine will actually
# receive
cat("\\.R$")
## \.R$

So the regex engine never sees two backslashes. It receives \.R$, and that is the pattern described below. Typing a single backslash does not work either: "\.R$" is a syntax error, because \. is not one of R’s recognised string escapes.

With that settled, the pattern reads character by character:

  • \. means a literal dot. In a regular expression a bare . means any character at all, so it has to be escaped to mean a full stop.
  • R is simply the letter R.
  • $ anchors the match to the end of the name.

Read as a whole: names that end in a dot followed by R. Both halves are doing work, and dropping either one breaks it in a different way:

  • Without the $, the pattern "\.R" matches anywhere in the name, so it also collects Chapter2_partC.Rmd, notes.Rdata and proj.Rproj. R would then try to source() your R Markdown document, which is not R code.
  • Without escaping the dot, the pattern ".R$" means any character, then R, at the end, so a file called myR with no extension at all would be collected too.

💡 The match is case-sensitive

A function saved as my_function.r is skipped in silence: no error, no warning, and the function simply is not there when you call it. Keep to a capital .R, or pass ignore.case = TRUE to list.files().

✅ Checkpoint

Your setup chunk sources your UDFs, and Chapter2_partC.Rmd knits without error.

A chunk below it calls square_number() and its output appears in the knitted document. If you get could not find function "square_number", the source() path is wrong relative to the .Rmd, not relative to wherever your console happens to be pointing.

Your functions are now in exactly one place. Edit square_number.R once and every document that sources it picks up the change on the next knit. That is the whole argument for UDFs: a calculation defined in one file, not copied into five analyses that then drift apart.

3.7 PART F: Interactive Tutorials

The objective of this section is to provide students with some tools and ideas to design their bioinformatic tutorials (for PART 2). Here, students will have an overview of the tools implemented in the R package learnr, which was developed to produce interactive tutorials. Although developed in R, the interactive tutorials are designed to be conducted in web browsers (but it could also be entirely done within RStudio).

The interactive tutorial presented here is subdivided into five topics:

  1. Introduction: This part sets the scene and provides some background information on the tutorial.
  2. Exercise: The aim of the exercise is presented together with a pseudocode (with associated R functions) outlying steps to design and implement an R code to complete the exercise. Students are asked to develop their R codes and execute them in the code chunk. You can type your code directly in the window/console and execute it using the R Code button. The instructor has also provided the solution to the exercise, which could be accessed by pressing the Solution button available in the top banner of the code chunk window.
  3. Questions: These questions are designed to test students’ knowledge of the code.
  4. Solution: A commented solution proposed by the instructor is available in a code chunk. Students can execute the code and inspect outputs produced to generate the final answer.
  5. YouTube tutorial: Here, a short YouTube video presents the procedure to launch a learn interactive tutorial and briefly presents the exercise. The main objective of this video is to show students that it is quite easy to integrate a video into their tutorials.

Finally, the instructor wants to stress that students are not obliged to design their tutorials using learnr. You can also use the R Markdown language/syntax and output tutorials in HTML or PDF formats (more on this subject in Chapter 1).

This document highlights steps to execute the interactive tutorial designed by the instructor.

3.7.1 Installation of required package

Open RStudio and install the learnr package from CRAN by typing the following command in the console:

install.packages("learnr")

3.7.2 Files location & description

Files associated to this tutorial are deposited on the Google Drive under this path: Reproducible_Science -> Bioinformatic_tutorials -> Intro_interactive_tutorial

There are two main files:

  • README.html: The documentation to install R package and run interactive tutorial.
  • EEB603_Interactive_tutorial.Rmd: The interactive tutorial written in R Markdown, but requiring functions from learnr.

3.7.3 YouTube video

The instructor has made a video explaining the procedure to launch the interactive tutorial (based on option 1; see below) as well as some additional explanations related to the exercise.

3.7.4 Running tutorial

  1. Download the Intro_interactive_tutorial folder and save it on your local computer.
  2. Set working directory to location of EEB603_Interactive_tutorial.Rmd.
  • This can be done as follows in RStudio: Session -> Set Working Directory -> Choose Directory....
  • You can also use the setwd() function to set your working directory (e.g. setwd("~/Documents/Course_Reproducible_Science/Timetable/Intro_interactive_tutorial")).
  1. Running tutorial. There are two options to run the integrative tutorial:
  • Option 1: Open EEB603_Interactive_tutorial.Rmd in RStudio and press the Run Document button on the upper side bar to launch the tutorial. It will appear in the Viewer panel (on right bottom corner). You can open the interactive tutorial in your web browser by clicking on the third icon at the top of the viewer panel. This procedure is also explained in the Youtube video.
  • Option 2: Type the following command in the console. This will open a window in RStudio. You can also open the tutorial in your web browser by pressing the Open in Browser button.
rmarkdown::run("EEB603_Interactive_tutorial.Rmd")
  1. Enjoy going through the tutorial!

3.7.5 Learning syntax to develop interactive tutorials

The procedure to develop interactive tutorials using learnr is presented here. To learn more about the syntax, the instructor encourages you to open EEB603_Interactive_tutorial.Rmd in RStudio and inspect the document. This will allow learning syntax and associated procedures to:

  • Design exercises (with embedded R code chunks and associated solutions).
  • Include multiple choices questions.
  • Embed a YouTube video.

4 Chapt. 3 – A Roadmap for Reproducible Science

4.1 Aim

The aim of this chapter is to engage in readings and group discussions that equip you with the knowledge and tools needed to implement reproducible science practices in support of your research.

We will begin by reviewing the steps of the scientific process and identifying potential threats to its integrity. Then, we will discuss strategies to mitigate these threats and explore initiatives that promote and reward reproducibility in science.

This chapter has a second purpose: it puts your own charter to the test. In Chapter 1 your group wrote a reproducibility charter, a short set of standards your research team binds itself to. You wrote it before you had any vocabulary for how research actually goes wrong. Both activities in this chapter send you back to that document: first to ask whether your standards would survive the specific threats described here, then to score them against what the research community now demands through the TOP Guidelines. A charter that is never tested is a wish-list.

4.2 Let’s Start with a Quote

While studying the material presented in this chapter, keep in mind this quote from Richard Feynman:

The first principle is that you must not fool yourself — and you are the easiest person to fool.

4.3 Learning Outcomes

By the end of this chapter, you will be able to:

This chapter runs over a single class, and it assumes you have done the reading. You read Discuss Solutions to Mitigate Threats and the TOP Guidelines at home beforehand. That reading is what makes the session work: both charter activities happen in this one class, and neither of them goes well if you arrive without knowing what the established mitigations are. My pass over the scientific process and the threats to it is a fifteen-minute synthesis from the slides, not your first contact with the material.

Block Time Who
Review the Steps Involved in the Scientific Process 8 min Instructor, from the slides
Identify and Define Threats to the Scientific Process 7 min Instructor, from the slides
👥 Do Your Standards Hold? 30 min Students, in 2 groups
five-minute break 5 min
The Transparency and Openness Promotion Guidelines 10 min Instructor
👥 Score Your Charter Against TOP 10 min Students, in 2 groups
Total in class 70 min
Discuss Solutions to Mitigate Threats 20 min Students, before class
📰 Compare With a Real Journal 10 min Students, before the next class
🤝 Report back on both activities 10 min Students, at the start of the next class

4.4 Class Outline

  1. Review the Steps Involved in the Scientific Process (8 minutes, from the slides)
  2. Identify and Define Threats to the Scientific Process (7 minutes, from the slides)
  3. Discuss Solutions to Mitigate Threats to the Scientific Process (20 minutes, read at home before class)
  4. 👥 Group Activity: Do Your Standards Hold? (30 minutes)
  5. The Transparency and Openness Promotion Guidelines (10 minutes)
  6. 👥 Group Activity: Score Your Charter Against TOP (10 minutes in class; the journal comparison is homework)
  7. 🤝 Report Back on Both Activities (10 minutes, at the start of the next class)

4.5 Resources

This chapter is primarily based on the following resources:

  • Kerr NL (1998). “HARKing: Hypothesizing After the Results are Known.” Personality and Social Psychology Review, 2(3), 196-217. https://doi.org/10.1207/s15327957pspr0203_4. – On HARKing (see Glossary section).
  • Nosek BA, Spies JR, Motyl M (2012). “Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability.” Perspectives on Psychological Science, 7(6), 615-631. https://doi.org/10.1177/1745691612459058. – On incentives.
  • Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, Buck S, Chambers CD, Chin G, Christensen G, Contestabile M, Dafoe A, Eich E, Freese J, Glennerster R, Goroff D, Green DP, Hesse B, Humphreys M, Ishiyama J, Karlan D, Kraut A, Lupia A, Mabry P, Madon T, Malhotra N, Mayo-Wilson E, McNutt M, Miguel E, Paluck EL, Simonsohn U, Soderberg C, Spellman BA, Turitto J, VandenBos G, Vazire S, Wagenmakers EJ, Wilson R, Yarkoni T (2015). “Promoting an open research culture.” Science, 348(6242), 1422-1425. ISSN 0036-8075, https://doi.org/10.1126/science.aab2374, http://science.sciencemag.org/content/348/6242/1422.full.pdf, http://science.sciencemag.org/content/348/6242/1422. – On promoting an open research culture.
  • Grant S, Corker KS, Mellor D, others (2026). “TOP 2025: An update to the Transparency and Openness Promotion Guidelines.” Research Integrity and Peer Review, 11, 40. https://doi.org/10.1186/s41073-026-00223-0. – TOP 2025, the current version of the guidelines: seven research practices, and three levels renamed to say who is responsible.
  • Baker M (2016). “1,500 scientists lift the lid on reproducibility.” Nature, 533(7604), 452-454. https://doi.org/10.1038/533452a. – Results of a reproducibility survey.
  • Munafo MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, du Sert NP, Simonsohn U, Wagenmakers E, Ware JJ, Ioannidis JPA (2017). “A manifesto for reproducible science.” Nature human behaviour, 1(0021), 0021. ISSN 2397-3374, https://doi.org/10.1038/s41562-016-0021. – Provides a framework to implement the measures proposed in this chapter.
  • Nosek BA, Ebersole CR, DeHaven AC, Mellor DT (2018). “The preregistration revolution.” Proceedings of the National Academy of Sciences, 115(11), 2600-2606. ISSN 0027-8424, https://doi.org/10.1073/pnas.1708274114, http://www.pnas.org/content/115/11/2600.full.pdf, http://www.pnas.org/content/115/11/2600. – On the pre-registration revolution.
  • Troudet J, Vignes-Lebbe R, Grandcolas P, Legendre F (2018). “The Increasing Disconnection of Primary Biodiversity Data from Specimens: How Does It Happen and How to Handle It?” Systematic Biology, syy044. https://doi.org/10.1093/sysbio/syy044, /oup/backfile/content_public/journal/sysbio/pap/10.1093_sysbio_syy044/3/syy044.pdf, http://dx.doi.org/10.1093/sysbio/syy044. – On the accuracy of biodiversity occurrence data available in GBIF.

The website of the Center for Open Science was also used to design the chapter content. More specifically, see this webpage on study pre-registration.

4.6 Teaching Material

The presentation associated with this class is available here:

💡 Best browser to view presentation

  • For optimal viewing of this presentation in full-screen presentation mode, the instructor recommends using Mozilla Firefox. Its built-in PDF viewer supports seamless full-screen display with easy navigation tools, making it ideal for presentations without the need for external plugins. Other browsers may not handle PDF presentation features as reliably.

4.7 Review the Steps Involved in the Scientific Process (8 minutes, from the slides)

The scientific process can be subdivided into six phases (Figure 4.1):

  1. Ask a question, review the literature, and generate a hypothesis1.
  2. Design the study.
  3. Collect data.
  4. Analyze data and test the hypothesis.
  5. Interpret results.
  6. Publish and/or conduct the next experiment.
A clockwise circular cycle of six numbered stages linked by grey chevron arrows: ask a question, review literature and generate hypothesis; design study; collect data; analyze data and test hypothesis; interpret results; and publish or conduct the next experiment, which feeds back into stage one.

Figure 4.1: Overview of the scientific process.

To facilitate understanding of the material in this chapter, the six phases of the scientific process are grouped into three categories that reflect the progress of a study:

  • Pre-study: includes phases 1 and 2
  • Study: includes phases 3 to 5
  • Post-study: includes phase 6

Distinguishing these categories is essential for ensuring the reproducibility and transparency of a study, and for avoiding common pitfalls described in the next section of this chapter.

For instance, treating the pre-study category as a distinct step in the scientific process promotes study pre-registration and helps prevent practices such as HARKing, p-hacking, and publication bias (see the Glossary section).

Recognizing the post-study category is equally important, as it encourages both pre- and post-publication review, thereby supporting improved dissemination and transparency of your research.

4.8 Identify and Define Threats to the Scientific Process (7 minutes, from the slides)

A hallmark of scientific creativity is the ability to recognize novel and unexpected patterns in data. However, a major challenge for scientists is remaining open to new and important insights while also avoiding the tendency to be misled by perceived structure in randomness.

This challenge is often driven by a combination of cognitive biases, including:

  1. Apophenia: the tendency to see patterns in random data.
  2. Confirmation bias: the tendency to focus on evidence that aligns with our expectations or preferred explanations.
  3. Hindsight bias (also known as the knew-it-all-along effect): the tendency to view events as having been predictable only after they have occurred.

These factors can easily lead us to false conclusions and therefore pose significant threats to the scientific process. Some of these threats—such as HARKing and p-hacking—are illustrated in Figure 4.2, and definitions are provided in the Glossary section.

The six-stage circular research cycle in black, with threats added in red beside each stage: failure to control for bias at stage one, low statistical power and lack of replication at study design, poor quality control at data collection, P-hacking at analysis and interpretation, and publication bias, outcome switching and no data sharing at publication; a red double-headed arrow labelled HARKing cuts across the circle from interpret results back to hypothesis generation.

Figure 4.2: Overview of the scientific process and threats preventing reproducibility of the study (indicated in red). Abbreviations: HARKing: hypothesizing after the results are known; P-hacking: data dredging.

4.8.1 Glossary of Common Threats to the Scientific Process

Below are definitions of common threats to the scientific process (see Figure 4.2):

HARKing: Hypothesizing After the Results are Known. This occurs when a post hoc hypothesis—that is, a hypothesis developed based on or informed by the observed results—is presented in a research report as if it were an a priori hypothesis (formulated before the study).

Outcome switching: The practice of changing the study’s outcomes of interest after seeing the results. For example, a researcher may measure ten possible outcomes but then selectively report only those that show statistically significant results, intentionally or unintentionally. This increases the likelihood of reporting spurious findings by capitalizing on chance, while ignoring negative or non-significant results.

P-hacking: Also known as “data dredging,” this involves conducting multiple statistical tests on data and focusing only on those with significant results, rather than pre-specifying a hypothesis and performing a single appropriate test. This misuse inflates the chance of false positives by finding patterns that appear statistically significant but have no real underlying effect.

Publication bias: Also called the file drawer problem, this refers to the tendency for studies with positive, novel, or significant results to be published more often than studies with negative results or replications. As a result, the published literature overrepresents positive findings and can give a misleading impression of the strength of evidence.

4.9 Discuss Solutions to Mitigate Threats to the Scientific Process (20 minutes, read before class)

💡 Read this section at home, between the two classes

This is the one part of the chapter you cover on your own. It is not background reading: the activity that opens the next class, 👥 Do Your Standards Hold?, asks you to write new clauses for your charter, and a clause is far stronger when it names an established practice than when you invent one. The practices are here.

You do not need to memorise it. Know roughly what is in each of the five groups below, so that you can find the right one quickly when your group needs it.

The aim of this section is to discuss solutions for addressing threats to the scientific process by presenting measures that ensure reproducibility and transparency.

Following the framework proposed by Munafo et al. (2017), the measures covered in this section are organized into five categories that promote research reproducibility and transparency. When possible, these categories include specific working themes designed to minimize the threats discussed earlier (see Figure 4.2).

  1. Methods
  2. Reporting and Dissemination
  3. Reproducibility
  4. Evaluation
  5. Incentives

These measures are not intended to be exhaustive, but they provide a broad, practical, and evidence-based set of actions that can be implemented by researchers, institutions, journals, and funders. They also serve as a roadmap for students when designing their thesis projects.

4.9.1 Methods

This section outlines key measures that can be implemented during the research process—such as study design, methodology, statistical analysis, and collaboration—to improve scientific rigor, reproducibility, and transparency.

We have organized the content into the following themes:

4.9.1.1 Protecting Against Cognitive Biases

There is a substantial body of literature on the difficulty of avoiding cognitive biases. An effective strategy to mitigate self-deception and unwanted biases is blinding. In some research contexts, participants and data collectors can be blinded to the experimental conditions to which participants are assigned, as well as to the research hypotheses. Data analysts can also be blinded to key parts of the dataset. For example, during data preparation and cleaning (see Chapters 6 and 7), the identity of experimental conditions or variable labels can be masked so that outputs are not interpretable in terms of the research hypotheses.

Pre-registration of the study design, primary outcome(s), and analysis plan (see the Promoting Study Pre-registration section below) is another highly effective form of blinding, as the data do not yet exist and the outcomes are not yet known.

4.9.1.2 Improving Methodological Training

Research design and statistical analysis are deeply interconnected. Common misconceptions—such as misunderstanding p-values, the limitations of null hypothesis significance testing, the importance of statistical power, the accuracy of effect size estimates, and the likelihood that a statistically significant result will replicate—can all be addressed through improved statistical training. These topics are covered in BIOL603: Advanced Biometry.

4.9.1.3 Improving the Chain of Evidence Supporting Biodiversity Occurrences

4.9.1.3.1 Are occurrences in GBIF scientifically sound?

Primary biodiversity occurrence data are central to research in Ecology and Evolution. However, these data are no longer collected as they once were. The mass production of observation-based (OB) occurrences is displacing the collection of specimen-based (SB) occurrences. Troudet et al. (2018) analyzed 536 million occurrences from the Global Biodiversity Information Facility (GBIF)2 and found that, between 1970 and 2016, the proportion of occurrences traceable to tangible material (i.e., SB occurrences) dropped from 68% to 18%. Moreover, many of the remaining specimen-based occurrences could not be readily traced back to a physical specimen due to missing information.

This alarming trend—characterized by low traceability and, therefore, low confidence in species identification—threatens the reproducibility of biodiversity research. For instance, low-confidence identifications limit the utility of large databases for deriving insights into species distributions, ecological traits, conservation status, phylogenetic relationships, and more.

Troudet et al. (2018) advocate that SB occurrences must continue to be collected to allow replication of ecological and evolutionary studies and to support rich, diverse research questions. When SB collection is not possible, they suggest OB occurrences should be accompanied by ancillary data (e.g., photos, audio recordings, tissue samples, DNA sequences). Unfortunately, such data are often not shared. A more rigorous approach to data collection—including proper documentation of ancillary evidence—can help ensure that recently collected biodiversity data remain useful and scientifically credible.

4.9.1.3.2 Specimens Are More Than Just Dead Stuff Stored in Dusty Cabinets

Specimens deposited in natural history museums and botanical gardens serve critical roles in biodiversity research. They enable:

  • Verification of species identifications
  • Access to ecological data from specimen labels (e.g., GPS coordinates, habitat type, soil composition)
  • Collection of morphological data (e.g., measuring phenotypic variation across a species’ range)
  • Sampling of tissues to support further analyses, including:
    • Molecular analyses (e.g., genotyping or sequencing)
    • Anatomical studies
    • Physiological analyses (e.g., stable isotope analysis for dietary inference)
    • etc.

These additional data are invaluable during the data collection phase and enhance the rigor of hypothesis testing. While SB occurrences may present logistical challenges (especially in field ecology), we strongly encourage students to collect and document ancillary data to support their observations and ensure the reproducibility of their analyses.

4.9.1.4 Implementing Independent Methodological Support

The need for independent methodological oversight is well-established in some fields. For example, many clinical trials employ multidisciplinary steering committees to oversee study design and execution. These committees arose in response to known financial conflicts of interest in clinical research.

Involving independent researchers—particularly methodologists who have no personal or financial stake in the research question—can reduce bias and improve study quality. This can be done either at the level of individual research projects or coordinated through funding agencies.

4.9.1.5 Encouraging Collaboration and Team Science

Studies of statistical power consistently find it to be low—often below 50%—across time and disciplines (see Munafo et al., 2017 and references therein). Low statistical power increases the likelihood of both false positives and false negatives, making it ineffective for building reliable scientific knowledge.

Despite this, low-powered research persists due to poor incentives, limited understanding of statistical power, and a lack of resources. Team science offers a solution: instead of relying on the limited capacity of individual investigators, distributed collaboration across multiple research sites supports high-powered study designs. It also improves generalizability across populations and contexts, and fosters integration of multiple theoretical frameworks and research cultures into a single project.

4.9.2 Reporting and Dissemination

This section describes measures that can be implemented when communicating research, including (for example) reporting standards, study pre-registration, and the disclosure of conflicts of interest.

We have organized the content into the following themes:

The last theme is addressed in more detail in Chapter 2.

4.9.2.1 Promoting Study Pre-registration

Progress in science relies, in part, on generating hypotheses from existing observations and testing hypotheses with new observations.
The distinction between postdiction3 and prediction is well understood conceptually, but often not respected in practice. Confusing postdictions with predictions reduces the credibility of research findings. Cognitive biases—particularly hindsight bias—make it difficult to avoid this mistake.

An effective solution is to define the research questions and analysis plan before observing the outcomes — a process known as pre-registration. Pre-registration clearly distinguishes between analyses and outcomes that result from a priori hypotheses and those that arise post hoc. Various practical strategies now exist to implement pre-registration effectively, even in cases involving pre-existing data. Services to support pre-registration are now available across disciplines, and their growing adoption is contributing to increased research transparency and credibility.

At its simplest, study pre-registration (see Nosek et al., 2018) may involve the basic registration of study design. More thorough pre-registration includes detailed specifications of procedures, outcomes, and the statistical analysis plan.

Study pre-registration addresses two major problems:

  1. Publication bias
  2. Analytical flexibility (especially outcome switching)

Definitions of these terms are provided in the Glossary section.

4.9.2.2 Improving the Quality of Reporting

Pre-registration enhances the discoverability of research, but discoverability does not guarantee usability.
Poor usability occurs when it is difficult to evaluate what was done, replicate the methods to assess reproducibility, or incorporate findings into systematic reviews and meta-analyses.

Improving the quality and transparency of research reporting is therefore essential. This includes using standardized reporting guidelines, ensuring that methods and data are clearly described, and making materials available for reuse wherever possible.

4.9.3 Reproducibility

This section describes measures that can be implemented to support the verification of research, including, for example, sharing data and methods.

4.9.3.1 Promoting Transparency and Open Science

Science is a social enterprise: independent and collaborative groups work to accumulate knowledge as a public good.
The credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data collected, the process of methodology implementation, data analysis, and outcome interpretation. Claims become credible through community review, criticism, extension, and reproduction of the supporting evidence. However, without transparency, credibility relies solely on trust in the confidence or authority of the originator. Transparency is superior to trust.

Open science refers to the process of making both the content and the process of producing evidence and claims transparent and accessible to others.
Transparency is a scientific ideal, and therefore adding ‘open’ should be redundant. In practice, however, science often lacks openness: many published articles are not available to those without personal or institutional subscriptions, and most data, materials, and code supporting research outcomes are not publicly accessible (though this is rapidly changing thanks to several initiatives, such as the Dryad digital repository).

Much of the research process—for example, study protocols, analysis workflows, and peer review—is historically inaccessible, partly because there were few opportunities to make it accessible, even if desired. This has motivated calls for open access, open data, and open workflows (including analysis pipelines). However, substantial barriers remain, including vested financial interests (particularly in scholarly publishing) and limited incentives for researchers to adopt open practices.

4.9.3.2 Promoting Open Science is Good, but the “Open” Costs Fall onto Researchers…

To promote open science, several open-access journals have recently been established (e.g., BMC, Frontiers, PLoS). These journals facilitate the sharing of scientific research—including associated methods, data, and code—but they are often expensive, with fees averaging over $1500 per publication.

Waivers may be available for researchers based in certain countries, and some institutions sponsor these initiatives, allowing a number of papers to be published for “free” each year. However, if you do not qualify for such waivers or institutional support, it can be challenging to cover these costs without grant funding. For example, the NSF is making efforts to promote open science through funding support.

The EEB program may be able to support some of these costs, but support will vary depending on the yearly budget and timing of the request. This topic is explored further in Chapter 4.

4.9.4 Evaluation

This section describes measures that can be implemented when evaluating research, including, for example, peer review.

4.9.4.1 Diversifying Peer Review: Pre- and Post-publication Reviews

For most of the history of scientific publishing, two functions have been confounded: evaluation and dissemination. Journals have provided dissemination by sorting and delivering content to the research community, and gatekeeping via peer review to determine what is worth disseminating. However, with the advent of the internet, individual researchers are no longer dependent on publishers to bind, print, and mail their research to subscribers. Dissemination is now easy and can be controlled by researchers themselves (see examples of preprint publishers below).

With the increasing ease of dissemination, the role of publishers as gatekeepers is declining. Nevertheless, the other role of publishing—evaluation—remains a vital part of the research enterprise. Conventionally, a journal editor selects a limited number of reviewers to assess the suitability of a submission for a particular journal. However, more diverse evaluation processes are now emerging, allowing the collective wisdom of the scientific community to be harnessed. For example, some preprint services support public comments on manuscripts, a form of pre-publication review that can be used to improve the manuscript (see below). Other platforms, such as PubMed Commons and PubPeer, offer public forums to comment on published works, facilitating post-publication peer review. At the same time, some journals are trialing ‘results-free’ review, where editorial decisions to accept are based solely on the rationale and study methods (i.e., results-blind; for instance, PLoS ONE applies this approach).

Both pre- and post-publication peer review mechanisms dramatically accelerate and broaden the evaluation process. By sharing preprints, researchers can obtain rapid feedback on their work from a diverse community, rather than waiting several months for a few reviews in the conventional, closed peer review process. Using post-publication services, reviewers can make positive and critical commentary on articles instantly, rather than relying on the laborious, uncertain, and lengthy process of authoring a commentary and submitting it to a publishing journal for possible publication.

4.9.4.2 Preprint Services to Disseminate Your Research Early and Receive Feedback

bioRxiv (pronounced “bio-archive”) is a free online archive and distribution service for unpublished preprints in the life sciences. It is operated by Cold Spring Harbor Laboratory, a not-for-profit research and educational institution. By posting preprints on bioRxiv, authors can make their findings immediately available to the scientific community and receive feedback on draft manuscripts before submitting them to journals.

Articles on bioRxiv are not peer-reviewed, edited, or typeset before being posted online. However, all articles undergo a basic screening process to check for offensive and/or non-scientific content, potential health or biosecurity risks, and plagiarism. No endorsement of an article’s methods, assumptions, conclusions, or scientific quality by Cold Spring Harbor Laboratory is implied by its appearance on bioRxiv. An article may be posted prior to, or concurrently with, submission to a journal but should not be posted if it has already been accepted for publication.

PeerJ Preprints is a preprint server for the Biological Sciences, Environmental Sciences, Medical Sciences, Health Sciences, and Computer Sciences. A PeerJ Preprint is a draft of an article, abstract, or poster that has not yet been peer-reviewed for formal publication. Authors can submit a draft, incomplete, or final version of their work for free.

Submissions to PeerJ Preprints are not formally peer-reviewed. Instead, they are screened by PeerJ staff to ensure that they fit the subject area, do not contravene any policies, and can reasonably be considered part of the academic literature. Submissions deemed unsuitable in any of these respects will not be accepted for posting. Content considered non-scientific or pseudo-scientific will not pass screening.

4.9.5 Incentives

Publication is the currency of academic science, influencing employment, funding, promotion, and tenure. However, not all research is equally publishable. Positive, novel, and clean results are more likely to be published than negative findings, replications, or results with loose ends. Consequently, researchers are incentivized to produce the former, sometimes at the expense of accuracy (see Nosek et al., 2012). These incentives ultimately increase the likelihood of false positives in the published literature. Changing these incentives offers an opportunity to enhance the credibility and reproducibility of published results.

Funders, publishers, societies, institutions, editors, reviewers, and authors all contribute to the cultural norms that create and sustain dysfunctional incentives. Therefore, shifting incentives requires a coordinated effort by all stakeholders to alter reward structures. While there will always be incentives for innovative outcomes—rewarding those who discover new things—there can also be incentives for efficiency and rigor. Researchers who conduct transparent, rigorous, and reproducible research should be rewarded more than those who do not.

Promising examples of effective interventions include journals adopting:

  • Badges to acknowledge open practices.
  • Registered Reports as a results-blind publishing model.
  • TOP (Transparency and Openness Promotion) guidelines to promote openness and transparency.

Collectively, and at scale, these efforts can shift incentives so that what benefits the scientist also benefits science—encouraging rigorous, transparent, and reproducible research that produces credible results.

4.10 👥 Group Activity: Do Your Standards Hold? (30 minutes)

In Chapter 1 your group wrote a reproducibility charter: a short set of standards your research team binds itself to. You wrote it before you had any vocabulary for the ways research actually goes wrong. You now have that vocabulary, so the charter can be tested.

💡 Your roles, and the new one

You are four to a group, and from today there are four roles, so everyone owns a section of their own.

  • Keep the role you took in Chapter 1: Data lead, Analysis lead or Reporting lead.
  • The fourth member becomes the Evaluation lead, who owns everything that happens once work leaves the group: preprints, peer review, and how the group answers reviewers. If a section had two co-leads in Chapter 1, one of them takes this role now.
  • Your charter has no Evaluation section, because Chapter 1 never raised the question. Today it gets one, written from scratch.

How the work divides. The diagnosis is done together; the writing is done alone. In Step 2 the whole group works through all four threats at one table, but each threat is pinned to the section that should have stopped it, and that section’s lead has to produce the clause or say out loud that there is none. In Step 3 you each write in your own section.

And the Evaluation lead is not a spectator. You own publication bias outright in Step 2, because deciding not to submit a null result is precisely “what happens once work leaves the group”. You also keep the table and hold everyone else to the Chapter 1 test: could an outsider check this? Then in Step 3 you draft the entire new Evaluation section, which is the largest single piece of writing in this activity.

Work in your Chapter 1 charter document, in our shared Drive at Reproducible_Science > Chapters > Chapter_1 > Part_B. You are revising a live document, not starting a new one.

4.10.1 🧭 Define All Four Threats (4 minutes)

Both groups take all four threats. You are testing your own charter, and there are two groups with two charters, so splitting the list would leave half of each charter untested. The four fall into two pairs, and the contrast is worth holding onto:

Pair Threats What they have in common
1 HARKing and P-hacking Decisions made after seeing the data
2 Outcome switching and Publication bias What gets reported, and what quietly does not

For each of the four, agree on:

  1. A one-sentence definition, from the Glossary of Common Threats. You read it before class, so this is a check rather than a first pass.
  2. Which step of the scientific process it strikes, and how it biases the result.

4.10.2 🔍 Test Your Charter Against All Four (18 minutes)

Work at one table, with the Evaluation lead keeping the record. Take each threat in turn and do two things: decide which section of your charter should have stopped it, then make that section’s lead either quote the clause or state plainly that there is none. Nobody diagnoses their own section in private, and nobody gets to hide behind the group.

Threat Whose section should stop it? The clause that would stop it (quote it) Verdict
HARKing
P-hacking
Outcome switching
Publication bias

Roughly four minutes each. If a threat is clearly covered, say so and move on: the ones worth your time are those where you have to argue about it.

💡 Expect to argue about the second column, and do not resolve it too quickly

Two of these threats land on the Analysis lead (HARKing and p-hacking are both about decisions made after seeing the data) and publication bias lands on the Evaluation lead. That leaves outcome switching genuinely contestable: is it the Analysis lead’s, for changing the measure, or the Reporting lead’s, for not disclosing the change?

Notice what is missing. None of the four is primarily a Data lead threat, which is worth a minute of your time rather than a shrug: either your data section is already the safest part of your charter, or these four threats simply do not reach the place where data are generated. Both answers are interesting, and the second one tells you what a fifth threat would have to be.

💡 Be hard on yourselves here

  • A clause only counts if somebody outside your group could check whether you followed it. “We will be transparent about our analysis” stops nothing. “The analysis plan is written and committed before the data are unblinded” stops p-hacking.
  • Give each threat one of three verdicts: covered, partly covered, or not covered.
  • Expect gaps. Finding them is the exercise; passing is not.

4.10.3 ✍️ Write the Missing Clauses (8 minutes)

For every threat you marked partly or not covered, write the clause that closes the gap. Use the solutions you read before class as your source: a clause is far stronger when it names an established practice (pre-registration, reporting standards, open data) than when you invent one.

  • If you found more than two gaps, write the two that would most change how your group actually works, and record the others in the charter as dated to-dos. Detecting every gap is the point of the previous step; closing them all is not today’s work.
  • Each new clause goes under the section whose lead owns it, marked with today’s date, so the charter records what you learned and when.
  • The Evaluation lead opens a new Evaluation heading and writes the group’s first standards for peer review and preprints.
  • Before you finish, read the new clauses aloud and ask the Chapter 1 question again: could an outsider check this?
  • Keep one or two of these clauses to hand. You will read them out, before and after, when you report back next class.

4.11 The Transparency and Openness Promotion Guidelines (10 minutes)

The Transparency and Openness Promotion (TOP) Guidelines (published in Nosek et al., 2015) provide a framework for journals and funders to encourage greater transparency in research planning and reporting.

The guidelines were revised in 2025 (Grant et al., 2026). The 2015 framework published in Nosek et al. (2015) had eight modular standards; TOP 2025 has seven research practices, and its three levels were renamed so that each one says who does what. Both the count and the level names changed, so material written against the older version no longer matches the current guidelines.

Each practice still has three levels of increasing stringency, allowing for gradual implementation of open science practices.

Journals can choose which standards to adopt and at what level, providing flexibility to account for disciplinary norms.

This structure helps establish community-wide transparency standards while respecting the diversity of research practices across fields.

4.11.1 The Seven TOP Research Practices

The three levels are the key to reading this table. They are no longer only about how strict a journal is; they say who is responsible at each step:

  • Level 1, Disclosed — the authors state whether they did it, and where.
  • Level 2, Shared and Cited — the authors do it, deposit the output and cite it.
  • Level 3, Certified — an independent party verifies that it was done properly.
Research practice Level 1: Disclosed Level 2: Shared and Cited Level 3: Certified
Study Registration Authors state whether the study was registered, and if so, where and when Researchers register the study and cite the registration An independent party certifies the registration was timely and complete for the design
Study Protocol Authors state whether the protocol is available, and if so, where and when it was shared Researchers publicly share the protocol and cite it An independent party certifies the protocol was shared on time and is complete
Analysis Plan Authors state whether the analysis plan is available, and if so, where and when Researchers publicly share the analysis plan and cite it An independent party certifies the plan was shared on time and is complete
Research Materials Transparency Authors state whether materials are available, and if so, where Researchers cite materials deposited in a trusted repository An independent party certifies the materials were deposited and documented to best practice
Analytic Code Transparency Authors state whether analytic code is available, and if so, where Researchers cite code deposited in a trusted repository An independent party certifies the code was deposited and documented to best practice
Data Transparency Authors state whether data are available, and if so, where Researchers cite data deposited in a trusted repository An independent party certifies the data were deposited with metadata to best practice
Reporting Transparency Authors state whether they used a reporting guideline, and if so, which one Authors publicly share a completed reporting checklist and cite the guideline An independent party certifies the appropriate guideline was followed

Journals select levels based on their readiness and disciplinary norms, balancing implementation feasibility against stronger transparency requirements.

💡 What happened to replication?

In the 2015 framework there were eight standards rather than seven, and Replication was the eighth. TOP 2025 moves it out of the research practices and into a separate set of verification study types, alongside registered reports, multiverse analyses and many-analyst studies. It also adds two verification practices: results transparency and computational reproducibility. That second one is this entire course: the claim that somebody else can re-run your analysis and obtain your numbers is now a named part of the framework rather than an implication of it.

Citation standards, the first of the 2015 standards, no longer has a row of its own. Citing the data, code and materials you use is now built into Level 2 of each practice, which is a stronger position: you cannot reach Level 2 on data transparency without citing a deposit.

Over 1,000 journals and organizations had adopted one or more TOP-compliant policies as of August 2018, including Ecology Letters, The Royal Society, and Science. Over 3,000 journal policies have now been scored.

4.11.2 Where to Look Up a Journal

The entry point is the Center for Open Science’s TOP Guidelines page, which links the current data. The three files you are most likely to want:

Source What it holds Last updated
top-factor.csv Over 3,000 journal policies scored against each standard and level. The most useful single file March 2025
TOP_signatories.csv Journals and organisations that have declared support for TOP February 2023
TOP_implementors.csv The older list of journals that had implemented a TOP policy June 2019

The scoring evaluation rubric explains how a policy was turned into a score, which is worth a look before you quote one.

💡 Two traps when you look up a journal

  • TOP Factor has been retired. The site topfactor.org now redirects to the guidelines page, so a search result or a slide pointing you there is out of date. The scored data above is what replaced it.
  • Everything in those files is scored against the 2015 eight-standard framework, because all of it predates TOP 2025. A journal’s own published policy will usually be in that older language too. So expect to translate: work out which of the seven current practices each old standard corresponds to, and say so when you report. That translation is part of the exercise, not an obstacle to it.

The current guidelines themselves are maintained at https://www.cos.io/initiatives/top-guidelines.

4.11.3 Promoting Linkage of Publication, Data, Code, and Analyses into a Unified Environment

The material in Chapter 1 on R Markdown (as used in RStudio) addresses the need for a unified environment that links publications, code, and data.

The Center for Open Science offers a platform called the Open Science Framework (OSF) to support this goal (see Figure 4.3). OSF is a free, open-source project management and repository platform designed to assist researchers throughout the entire project life cycle.

As a collaboration tool, OSF allows researchers to work privately with selected collaborators or to make parts or all of their projects publicly accessible, with citable and discoverable DOIs for broader dissemination.

As a workflow system, OSF integrates with many existing research tools, enabling researchers to manage their entire projects from one place. This connectivity helps eliminate data silos and information gaps by allowing tools to work together seamlessly, reflecting how researchers actually collaborate and share knowledge (see Figure 4.3).

A circular wheel of research lifecycle stages running clockwise from search and discover, develop idea, design study, acquire materials, collect data, store data, analyze data, interpret findings, write report and publish report, with the OSF logo at the hub; logos of third-party services ring the outside next to the stage each supports, including Zotero and Mendeley for discovery, GitHub, Dropbox and Google Drive for data storage, and Zenodo, figshare and Dryad for publication.

Figure 4.3: Overview of the OSF workflow and connections with other widely used software.

4.12 👥 Group Activity: Score Your Charter Against TOP (10 minutes in class)

You have just asked whether your charter stops specific threats. Now a harder question: how does it compare with what the research community already demands? The TOP Guidelines are the closest thing to a published version of what your group wrote, agreed across thousands of journals, and they come with levels, so you can score yourselves instead of arguing.

4.12.1 📊 Score Yourselves (10 minutes)

Score in parallel, then reconcile. Seven practices is too many for one conversation in ten minutes, so each lead scores the practices that belong to their own section first, and the group then pools the results and argues about the ones that do not fit:

Lead Practices you score
Data lead Study Protocol, Research Materials Transparency
Analysis lead Analysis Plan, Analytic Code Transparency
Reporting lead Data Transparency, Reporting Transparency
Evaluation lead Study Registration, and you keep the table and audit the other three

Two of these allocations are worth a word. Data Transparency is the Reporting lead’s, not the Data lead’s, because Chapter 1 gave them “what is deposited and when, and how data, software and sources are cited”, which is Level 2’s requirement in your own words. Study Registration is the Evaluation lead’s, because a registered report is submitted and reviewed before the results exist, which is squarely what happens once work leaves the group.

For each practice, decide which level your charter currently meets, judging the charter as written rather than as intended:

  • Not addressed — the charter says nothing about it. This is not a TOP level; TOP starts at Level 1. It is simply where a charter can be, and it is the most useful thing this exercise finds.
  • Level 1, Disclosed — you commit to stating whether you did it, and where.
  • Level 2, Shared and Cited — you commit to doing it, depositing the output and citing it.
  • Level 3, Certified — you commit to having it checked by somebody outside the group.

Timing. Roughly 3 minutes scoring your own, 6 minutes pooling them into one table and arguing the contested ones, 1 minute to date it and save.

Record the result as a seven-row table at the top of your charter, with today’s date. A charter that reads “Not addressed” most of the way down is a perfectly respectable result for a document written in week two. It is also a list of what to write next, in priority order.

💡 Evaluation lead: note the two practices that are not on the list

TOP 2025 also defines two verification practices: results transparency and computational reproducibility. They sit outside the seven, so you cannot score your charter against them today, and they are yours. Write them at the bottom of the table as your group’s stretch target, because computational reproducibility is what the rest of this course teaches you to do: by December you should be able to claim it rather than aspire to it.

4.12.2 📰 Compare With a Real Journal (10 minutes, before the next class)

Each member takes a different journal they might realistically publish in, and finds its TOP policy. Science is a worked example: https://www.science.org/content/page/science-journals-editorial-policies#TOP-guidelines

For your journal, identify:

  • which of the seven practices it addresses, and at what level (if its policy is written in the 2015 language, say which practice each old standard maps onto);
  • one place it is stricter than your charter;
  • one place your charter is stricter than it.

That second one matters. Journal policy is a floor, not a ceiling, and a group is free to hold itself to more than its publisher asks.

4.13 🤝 Report Back on Both Activities (10 minutes, at the start of the next class)

You do not report back today. Each group presents at the start of the next class, taking five minutes. That gap is deliberate: it gives you the evening to finish the journal comparison and to say what you did about what you found, rather than describing a gap you noticed ten minutes earlier.

This report draws on both of today’s activities:

Have the charter open and read from it, so that what we hear is your own wording.

4.13.1 What to report

Comes from What you give us Time
1 Do Your Standards Hold? One threat your charter already handled 1 min
2 Do Your Standards Hold? One or two clauses you changed, before and after 2 min
3 Score Your Charter Against TOP and the journal comparison Your highest and lowest practice, and the next clause you would write 1.5 min
4 Both One sentence on where the charter stands now 30 s

4.13.1.1 One threat your charter already handled (1 minute, from Testing Your Charter)

Name the threat, read the clause that stops it, and say who outside your group could check whether you followed it.

4.13.1.2 One or two clauses you changed (2 minutes, from Writing the Missing Clauses)

This is the centre of the report. For each clause, four things:

  • the threat or TOP practice that exposed the gap;
  • how the clause read before, or “there was nothing there”;
  • how it reads now;
  • what somebody outside the group would have to do to check it.

For example, a clause that moved from “we will be transparent about our analysis” to “the analysis plan is committed to the repository before the data are unblinded” shows the whole arc: a threat identified, a vague commitment found, and a checkable one written in its place.

4.13.1.3 Your TOP result (1.5 minutes, from Score Yourselves and the journal comparison)

  • the practice where you scored highest, and the clause that earned it;
  • the practice where you scored lowest or Not addressed, and the one clause you would write to move it up a single level;
  • one place the journal is stricter than you, and one place you are stricter than the journal.

4.13.1.4 One sentence to finish (30 seconds)

What would you tell somebody joining your group next week about what your charter now commits them to?

5 Chapt. 4 – Open Science and CARE Principles

5.1 Challenge

Today, funding agencies and journals are increasingly promoting Open Science, a topic we began discussing in Chapter 3. In this context:

Balancing the promotion of Open Science to support reproducibility with the values of your stakeholders can be a challenging task.

5.2 Aim

This chapter aims to explore how Open Science is promoted while recognizing when other principles, like the CARE Principles, must be applied to respect diverse perspectives, values, and data sovereignty.

It also provides solutions to address threats to the scientific process (introduced in Chapters 1 and 3) and guidance on managing the data life cycle and responsibly sharing research.

5.3 Learning Outcomes

By the end of this chapter, you will be able to:

This chapter runs over two classes, and most of it is group work. Three activities run back to back, which only works if you arrive having read the two frameworks they use: that reading is the homework, and it is listed below. The first class takes the FAIR activity as far as the pairs’ work; the FAIR report back opens the second class, followed by the CARE pass and the CARE activity.

Before class

Block Time Who
FAIR Data and The CARE Principles, including Data Sovereignty 25 min Students, at home

Class 1 — Open Science, and the FAIR Principles

Block Time Who
🤝 Present both Chapter 3 activities 10 min Students, in 2 groups
What is Open Science? 4 min Instructor
Quick pass over FAIR Data 3 min Instructor
👥 FAIR Principles Activity 15 min Students, in 4 pairs
five-minute break 5 min
Total in class 37 min

Class 2 — the CARE Principles, at the start of the Chapter 5 PART A class

Block Time Who
👥 FAIR pairs report back 4 min Students, in 4 pairs
Quick pass over CARE and Data Sovereignty 3 min Instructor
👥 CARE Principles Activity 15 min Students, in 4 pairs
Total 22 min

Not run in class

Block Time Who
👥 FAIR & CARE Comparison 10 min Reading, and a short written addition to your charter
Open Access and Creative Commons Licenses 15 min Instructor, in Chapter 6’s licence block

Open Access and Creative Commons Licenses are taught in Chapter 6’s licence block, where they sit naturally beside getting published.

The FAIR & CARE Comparison is not run in class this year. The material stays here and is worth reading: each pair already owns one FAIR and one CARE principle, so the comparison is something you can write in a few minutes rather than discuss.

5.4 Class Outline

  1. 🤝 Present both Chapter 3 activities (10 minutes)
  2. What is Open Science? (4 minutes)
  3. Quick pass over FAIR Data (3 minutes)
  4. 👥 FAIR Principles Group Activity (15 minutes)
  5. Quick pass over the CARE Principles and Data Sovereignty (3 minutes)
  6. 👥 CARE Principles Group Activity (15 minutes)
  7. 👥 FAIR & CARE Principles Comparison Activity (10 minutes)
  8. Open Access: A Tool to Promote Open Science? (8 minutes, taught in Chapter 6’s licence block)
  9. Creative Commons Licenses (7 minutes, taught in Chapter 6’s licence block)

5.5 Resources

This chapter is based on:

Web resources

Publications

Everything below is reading, and the three group activities at the end of the chapter apply it. Read FAIR Data, the CARE Principles and Data Sovereignty before class: I go over each of them briefly, immediately before the activity that uses it, so the pass in class is a reminder rather than your first contact.

5.5.1 What is Open Science?

The Organisation for Economic Co-operation and Development (OECD) defines Open Science as:

“to make the primary outputs of publicly funded research results – publications and the research data – publicly accessible in digital format with no or minimal restriction.”

While this definition focuses on access to research outputs, several other organizations advocate for a broader view of Open Science. According to the FOSTER Open Science initiative, Open Science involves:

“extending the principles of openness to the whole research cycle, fostering sharing and collaboration as early as possible, thus entailing a systemic change to the way science and research is done.”

5.5.2 What are the Pillars of Open Science?

The 7 pillars of Open Science are:

5.5.2.1 FAIR Data

The FAIR Principles (Wilkinson et al., 2016) guide researchers in ensuring that research outputs are (Figure 5.1):

  • Findable: Research outputs should be discoverable by the broader academic community and the public.
  • Accessible: Data should be accompanied by unique identifiers, metadata, and clear language, using standardized access protocols.
  • Interoperable: Data and metadata should follow recognized standards that allow them to be shared and integrated across systems, so that different tools and platforms can combine them without a human translating between them first (see What interoperability actually means).
  • Reusable: Research outputs should be designed for reuse and re-purposing, allowing others to maximize their research potential.

💡 Why FAIR Matters:

  • Reduces barriers to accessing and using research outputs.
  • Helps secondary researchers find, understand, and reuse data to generate new insights.
  • Maximizes the value of existing research resources.

💡 What interoperability actually means

Of the four, this is the one that sounds like jargon, so here is the problem it solves. You have a spreadsheet of plant occurrences and a colleague has another. Yours has a column called species, theirs calls it taxon, and a third lab calls it sci_name. Your dates read 12/10/2026, theirs Oct 12 2026. Your coordinates are in one column as 43.6N 116.2W, theirs in two columns of decimal degrees.

Nothing here is wrong, and nothing can be merged. Combining the two datasets means somebody sits down and translates one into the other, by hand, making judgement calls nobody records. Do that across a hundred labs and the cost is enormous; do it badly once and the errors are invisible afterwards.

Interoperability is the agreement that avoids that translation: everybody writes the same thing the same way, decided in advance and published, so that two datasets can be put side by side by a machine.

💡 The standard your field already uses: Darwin Core

Darwin Core, maintained by Biodiversity Information Standards (TDWG), is the vocabulary that biodiversity data are shared in. It fixes the spelling of each field once:

What you are recording Darwin Core term
The scientific name scientificName
When it was observed or collected eventDate
Where, in decimal degrees decimalLatitude, decimalLongitude
Whether it is a specimen, an observation or a sequence basisOfRecord
The identifier of this particular record occurrenceID

This is why GBIF works at all. Billions of records, from thousands of museums, herbaria and field projects, can be searched together because each one arrives in the same vocabulary. Without it, GBIF would be a directory of incompatible spreadsheets.

And notice how the vocabulary satisfies the guidelines below. Each term has a persistent identifier of its own, for example http://rs.tdwg.org/dwc/terms/scientificName, so a machine can look up what the field means rather than guess from its name. That is I1 (a shared language) and I2 (vocabularies that are themselves FAIR) in practice, not in the abstract.

Four black-and-white icons in a row, each paired with a word: a magnifying glass for Findable, a hand tapping a screen for Accessible, three interlocking gears for Interoperable, and a recycling triangle for Reusable.

Figure 5.1: The FAIR Principles (source: SangyaPundir, CC BY-SA 4.0).

5.5.2.1.1 The FAIR Guiding Principles

Wilkinson et al. (2016) outline a set of guidelines that support the implementation of the FAIR Principles. These guidelines ensure that data and metadata are structured in a way that promotes discovery, access, integration, and reuse.

  1. To be Findable:
    • F1. (Meta)data are assigned a globally unique and persistent identifier.
    • F2. Data are described with rich metadata (as defined by R1 below).
    • F3. Metadata clearly and explicitly include the identifier of the data they describe.
    • F4. (Meta)data are registered or indexed in a searchable resource.
  2. To be Accessible:
    • A1. (Meta)data are retrievable by their identifier using a standardized communication protocol.
      • A1.1. The protocol is open, free, and universally implementable.
      • A1.2. The protocol allows for authentication and authorization, where necessary.
    • A2. Metadata remain accessible even when the data are no longer available.
  3. To be Interoperable:
    • I1. (Meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation.
    • I2. (Meta)data use vocabularies that themselves follow FAIR principles.
    • I3. (Meta)data include qualified references to other (meta)data.
  4. To be Reusable:
    • R1. (Meta)data are richly described with a plurality of accurate and relevant attributes.
      • R1.1. (Meta)data are released with a clear and accessible data usage license.
      • R1.2. (Meta)data are associated with detailed provenance.
      • R1.3. (Meta)data conform to domain-relevant community standards.

5.5.2.2 Research Integrity

Research integrity refers to the commitment of researchers to act honestly, reliably, respectfully, and to be accountable for their actions throughout the research process.

At Boise State University, the Office of Research Compliance provides resources and guidance to support best practices in research integrity.

5.5.2.3 Next Generation Metrics

The Next Generation Metrics pillar of Open Science aims to shift cultural thinking around how research is evaluated. It encourages moving beyond traditional metrics—such as citation counts and journal impact factors—toward more diverse, meaningful indicators of research quality and influence.

These metrics, drawn from a variety of sources and describing different aspects of scholarly impact, can help us gain a broader and more nuanced understanding of the significance and reach of research.

In this chapter, we demonstrate this topic with two examples:

5.5.2.3.1 The San Francisco Declaration on Research Assessment

The San Francisco Declaration on Research Assessment (DORA) exemplifies the growing shift in how research quality is evaluated. Increasingly, institutions and funding bodies are supporting DORA and openly rejecting the use of journal impact factors and other quantitative metrics as primary indicators of research quality.

Among many others, Springer Nature has endorsed DORA. You can view their specific commitments here.

5.5.2.3.2 Altmetric

Another example is Altmetric, a tool that tracks and demonstrates the reach and influence of research outputs—particularly among key stakeholders—by focusing primarily on social media and other online platforms (Figure 5.2).

Screenshot of a journal article Metrics panel showing 1,062 total views (781 pageviews, 281 PDF downloads), a red line chart of monthly views peaking above 300, a Dimensions citation badge reading 2 total citations, and an Altmetric donut of 22 broken down into 1 news outlet, 1 blog, 10 tweets, and 7 Mendeley readers.

Figure 5.2: Example of new metrics associated to publications to assess impact of research beyond the impact factor.

5.5.2.4 Future of Scholarly Communication

The future of scholarly communication is one of the most prominent pillars of Open Scholarship, as it aims to shift the current academic publishing model toward fully Open Access. We will explore this topic further in the section on Open Access and associated licenses below.

5.5.2.5 Citizen Science

Citizen Science refers to the increased involvement of the public in the research process, recognizing the invaluable insights that non-academic contributors can offer. These contributions often provide perspectives or data that researchers might not otherwise access. Examples of such initiatives include eBird and iNaturalist.

Leveraging the internet, openly available tools, and local knowledge, citizen science is transforming how research is conducted. No longer limited to academic researchers, it encourages collaboration across society.

Wagenknecht et al. (2021) discuss this concept through a European Citizen Science project, while Groom et al. (2017) reflect on its role in biodiversity research. Citizen science has the potential to significantly enhance your research—but it’s important to be aware of its challenges. For more on this topic, refer to this section in Chapter 3.

5.5.2.6 Education and Skills

This pillar focuses on identifying researchers’ training needs and addressing gaps in knowledge and skills related to Open Science—such as making publications openly accessible, managing research data in alignment with the FAIR Principles (Figure 5.1), and upholding research integrity.

All researchers, regardless of career stage, should have access to training and professional development opportunities that support Open Science practices. Additionally, skills development should be extended to other stakeholders in the research ecosystem, including librarians, data managers, and the public, particularly in support of citizen science initiatives.

At Boise State University, the Albertsons Library offers resources, workshops, and seminars on these topics.

5.5.2.7 Rewards and Initiatives

Promoting engagement with Open Science requires recognizing and rewarding the efforts of those who contribute to it. This pillar addresses systemic barriers while advocating for best practices.

A common barrier is the perceived lack of recognition for tasks such as managing research data or publishing open access. These efforts are often undervalued in traditional academic assessments, discouraging wider adoption of Open Science practices.

Work under this pillar seeks to overcome these challenges and promote greater participation in Open Science. For more on current initiatives and reward systems, refer to Chapter 3.

5.5.3 The CARE Principles and Data Sovereignty

In this section, we investigate both the CARE Principles and Data Sovereignty, emphasizing how these concepts are closely connected.

As noted by Carroll et al. (2021):

“while big data, open data, and Open Science increase access to complex and large datasets for innovation, discovery, and decision-making, Indigenous Peoples’ rights to control and access their data within these environments remain limited.”

Integrating the FAIR Principles for scientific data with the CARE Principles for Indigenous Data Governance enhances machine actionability while prioritizing people and purpose. This combined approach supports Indigenous Peoples’ rights and interests in their data throughout the data lifecycle (Figure 5.3).

Teal graphic with two rounded panels linked by circuit-board style traces: the upper panel spells FAIR with Findable, Accessible, Interoperable, Reusable beneath each letter, the word 'and' sits between them, and the lower panel spells CARE with Collective Benefit, Authority to Control, Responsibility, Ethics beneath each letter.

Figure 5.3: Be FAIR and CARE (source: https://www.gida-global.org/care).

5.5.3.1 The CARE Principles for Indigenous Data Governance

The CARE Principles emphasize the importance of people and purpose in data governance, particularly for Indigenous communities. CARE stands for:

  • 🧩 Collective Benefit: Data ecosystems should be designed and operated in ways that enable Indigenous Peoples to derive meaningful benefits from data. This includes supporting governance, innovation, and community development.
  • 🛡 Authority to Control: Indigenous Peoples have the right to control the creation, collection, access, analysis, interpretation, management, dissemination, and reuse of data relating to their people, lands, and resources.
  • 📣 Responsibility: Researchers and data stewards must take responsibility for how Indigenous data is used, ensuring ethical practices and supporting capacity building within Indigenous communities.
  • ⚖️ Ethics: The use of Indigenous data must prioritize the rights, dignity, and well-being of Indigenous Peoples. This goes beyond formal compliance to include meaningful engagement and respect.

📝 The CARE Principles complement the FAIR Principles by adding a crucial human- and rights-centered lens to data governance.

Finally, the United Nations Declaration on the Rights of Indigenous Peoples affirms Indigenous Peoples’ rights and interests in data and their authority to control it. Access to data for governance is essential to support self-determination, and Indigenous nations should be actively involved in the governance of data to ensure ethical data use and reuse.

5.5.3.2 Data Sovereignty

The CARE Principles (Figure 5.3) are directly connected to the concept of data sovereignty. We provide a definition here because it is especially important for this course, as data sovereignty can impact your data sharing protocols. More on this topic will be discussed in Chapter 5.

Data sovereignty is the idea that data are subject to the laws and governance structures of the nation where they are collected. This concept is closely linked with data security, cloud computing, network sovereignty, and technological sovereignty. Unlike technological sovereignty, which is vaguely defined and often used as an umbrella term in policymaking, data sovereignty specifically concerns the governance of data itself.

Data sovereignty is usually discussed in two contexts: Indigenous groups and their autonomy from post-colonial states, and the control of transnational data flows. With the rise of cloud computing, many countries have enacted laws governing data control and storage, reflecting measures of data sovereignty. More than 100 countries now have data sovereignty laws in place.

In the context of self-sovereign identity (SSI), individuals can fully create and control their credentials, although nations may still issue digital identities within this framework.

5.5.4 Open Access: A Tool to Promote Open Science?

Open Access (OA) refers to a set of principles and practices that make research outputs freely available online, without access charges or other barriers (Figure 5.4). While OA publications are free for users to read, researchers often pay publication fees—typically between $1,500 and $2,500—to make their work openly accessible.

According to the 2001 definition of Open Access, it also involves removing or reducing barriers to copying and reuse by applying an open license. An open license allows others to reuse a creator’s work freely, in ways that are typically restricted by copyright, patent, or commercial licenses.

Most open licenses are:

  • Worldwide in scope
  • Royalty-free
  • Non-exclusive
  • Perpetual (they do not expire)

These characteristics ensure that research can be reused, adapted, and redistributed in alignment with the goals of Open Science.

5.5.4.1 The Open Access Conundrum

Since most Open Access (OA) journals rely on publication fees paid by authors to generate revenue, there is concern that some OA publishers may be motivated to accept lower-quality papers and forgo rigorous peer review in order to maximize profits.

At the same time, publication fees in prestigious OA journals can exceed $5,000, making this publishing model inaccessible for many researchers, particularly those without institutional funding or support.

This situation has been described as the “Open Access sequel to the Serials Crisis” (Khoo, 2019). The original Serials Crisis refers to the unsustainable pressure on library budgets caused by the rapid inflation of journal subscription costs. Open Access was initially proposed as a solution to this problem—by removing subscription barriers, research would remain accessible without ongoing costs. However, as Khoo (2019) notes, both traditional subscription models and Open Access publishing present their own challenges and limitations.

5.5.5 Creative Commons Licenses

Most Open Access journals use Creative Commons (CC) licenses, so it is important that you and your co-authors understand their implications before submitting your manuscript—especially regarding potential commercial use of your work by third parties (Figure 5.4).

Creative Commons defines the suite it maintains on its own home page:

The CC licenses and public domain tools give everyone, from individuals to large institutions, a standardized way to grant the public permission to use their work under copyright law.

Two words in that sentence carry most of the weight. Standardized is why a journal can name a licence in a single line and every reader in the world already knows what it permits, with no negotiation and no lawyer. Under copyright law is why a CC licence is not an alternative to copyright but an instrument of it: the licence works by exercising rights you already hold, which is also why you can only license what you own. And the suite covers more than the licences: the public domain tools are CC0, with which you waive your rights entirely, and the Public Domain Mark, which labels work already free of copyright.

A CC license is a type of public copyright license that enables the free distribution of a copyrighted work. It allows authors to grant others the right to share, use, and build upon their work while retaining some control over how it is used.

Creative Commons licenses provide flexibility; for example, authors can choose to allow only non-commercial uses. They also protect users and redistributors from copyright infringement claims, provided they follow the license terms set by the author.

A CC licence is the right instrument for the article and for the data, and the wrong one for the code. Software is licensed under a software licence instead, which settles questions a CC licence does not address, such as what happens to derived works and whether any warranty is given. So a single submission can carry three licences: one on the paper, one on the deposited data, and one on the repository holding the code.

To help authors select the most appropriate license for their work, the Creative Commons initiative offers an online tool, which I recommend consulting. For more detailed information on CC licenses, see this Wikipedia page.

Screenshot of a journal masthead in teal type with an Open Access badge, listing the editors-in-chief, the online ISSN and Wiley copyright, and a closing note that articles accepted from 14 August 2012 carry a Creative Commons Attribution License while earlier ones carry an Attribution Non-Commercial License.

Figure 5.4: Front page of Ecology and Evolution, an open access journal publishing articles under the Creative Common licensing system.

5.5.5.1 Available CC Licenses

The Creative Commons licenses provided by the CC initiative are summarized in Figure 5.5.

Dark slide titled Understanding CC Data License Options, with a vertical arrow on the left shading from green at the top labeled Most Open down to red at the bottom labeled Most Restrictive; alongside it a list running CC0 No Rights Reserved, CC BY, CC BY-SA, CC BY-NC-SA, and -ND NoDerivatives, matched on the right by the corresponding badge icons.

Figure 5.5: Overview of Creative Common licenses sorted from most open to most restrictive (provided by Andrew Child).

The description of these licenses is always presented in this format:

  1. You are free to:
    • Share — copy and redistribute the material under specified conditions.
    • Adapt — remix, transform, and build upon the material under specified conditions.
  2. Under the following terms:
    • Attribution — give appropriate credit to the original author.
    • No additional restrictions — you may not apply legal terms or technological measures that restrict others from doing anything the license permits.

Here is a list of some of the most commonly used CC licenses with URLs leading to their descriptions:

5.5.5.2 ShareAlike, and What Happens Without It

ShareAlike (SA) is the only clause that says anything about the licence on the next version of your work, and it is the Creative Commons equivalent of copyleft in software. CC BY-SA 4.0 requires anybody who remixes, transforms or builds upon your work to release that adaptation under the same licence. The permissions you granted travel with the work into every version downstream, which is the same logic as the GNU licences you will meet under Software Licenses.

Without SA, nothing travels. Under plain CC BY 4.0 somebody can take your figure, adapt it, and publish the adaptation under any terms they choose: all rights reserved, paywalled, no further reuse. They owe you attribution and nothing more. Your own version stays CC BY, because the licence you granted cannot be revoked, but the adapted version that circulates is theirs and it can be closed. CC BY is what most journals offer and most funders require, so this is a consequence you should accept knowingly rather than discover afterwards.

Figure 5.1 in this chapter is a worked example. It is CC BY-SA 4.0, which is why the caption names both the author and the licence, and why a redrawn version of it would have to be CC BY-SA 4.0 as well.

Two things SA is often confused with:

  • SA is not a ban on making money. CC BY-SA permits commercial use, so a company may sell a product built on your work provided the adaptation stays CC BY-SA. The clause that forbids commercial use is NonCommercial (NC), and it is separate (Figure 5.5). If your concern is somebody charging for your work, NC is the clause you mean, and note that it also stops a colleague putting your figure in a paid textbook or course pack.
  • SA is sticky, and that constrains you too. Because an adaptation has to keep the licence, you cannot redraw a CC BY-SA figure for a paper that your journal will publish under CC BY. Check the licence of every borrowed figure before you build on it, not at proof stage.

5.6 🤝 Group Sharing and Discussions

The three activities below are the core of this chapter. They run back to back, and each one builds on the last, so form your pairs once at the start and stay in them throughout.

💡 Read these three sections before class

These activities apply the two frameworks rather than introduce them, so a pair that arrives without having read the definitions spends its fifteen minutes looking them up instead of thinking about them. The sections are short.

5.6.1 👥 Your Pair, and the Principles It Owns

Work in four pairs. Both frameworks have exactly four components, so each pair owns one of each and keeps them for all three activities.

Pair FAIR principle CARE principle
Pair 1 Findable Collective Benefit
Pair 2 Accessible Authority to Control
Pair 3 Interoperable Responsibility
Pair 4 Reusable Ethics

The pairings are arbitrary, and that is the point: in the third activity you compare the two principles you happen to own, which is how these frameworks meet in practice. Nobody gets to choose a convenient pair.

5.6.2 👥 FAIR Principles Group Activity (15 minutes)

5.6.2.1 🎯 Learning Objective

Students collaboratively explore how the FAIR Principles apply to real-world research data and reflect on challenges and best practices for making data FAIR.

5.6.2.2 👥 Instructions

  1. Work in your pair, on your assigned FAIR principle (which one is mine?).
  2. Each of you should:
    • Re-read the definition of your principle in The FAIR Guiding Principles.
    • Think of a dataset (a survey, lab experiment results, environmental monitoring) and consider:
      • How would you apply this principle to that dataset?
      • What challenge or barrier might arise?
  3. Together, agree a 1-minute summary containing:
    • one key insight or challenge;
    • one tip or best practice for making data more FAIR.
  4. Report to the class. Four pairs, one minute each, so four minutes in total: keep to it.

5.6.3 👥 CARE Principles Group Activity (15 minutes)

5.6.3.1 🎯 Learning Objective

Students collaboratively explore how the CARE Principles apply to data involving Indigenous Peoples and reflect on ethical, cultural, and governance considerations in data stewardship.

5.6.3.2 👥 Instructions

  1. Same pairs, now on your assigned CARE principle (which one is mine?).
  2. Each of you should:
    • Re-read the definition of your principle in The CARE Principles for Indigenous Data Governance.
    • Consider a dataset or scenario involving Indigenous data (health records, environmental monitoring, traditional knowledge) and discuss:
      • How would this principle apply to it?
      • What challenges or considerations might arise?
  3. Together, agree a 1-minute summary containing:
    • one key insight or challenge;
    • one recommendation for respecting this principle in research or data governance.
  4. Report to the class, again one minute per pair.

5.6.4 👥 FAIR & CARE Principles Comparison Activity (10 minutes)

5.6.4.1 🎯 Learning Objective

Students critically compare the FAIR and CARE Principles and explore how they can be applied together to support ethical, inclusive, and technically sound data practices.

5.6.4.2 👥 Instructions

You now have two principles you know well: your FAIR one and your CARE one. This activity is about the relationship between those two specifically, so each pair is working on something different.

  1. Still in your pair, take the two principles you own, the ones you have spent the last half hour on, and answer:
    • Do they pull in the same direction, or against each other?
    • Think of a dataset involving sensitive or Indigenous data. Applying both of your principles to it, what has to be decided that neither principle settles on its own?
    • If they conflict, which one would you let win, and on what grounds?
  2. Prepare a 1-minute summary giving:
    • the relationship you found between your two principles;
    • one practical suggestion for holding both at once.
  3. Report to the class. Because each pair compared a different combination, the four reports together map most of how FAIR and CARE interact.

💡 The honest answer is sometimes “they conflict”

FAIR pushes data towards being open and machine-readable. CARE asks who has the authority to decide that, and in whose interest. Some pairs of principles genuinely cannot both be satisfied, and saying so, with reasons, is a better answer than a comfortable one that pretends the tension away.

6 Chapt. 5 – Data Management and Reproducible Code

6.1 Introduction

This chapter is divided into two parts:

6.2 Data Management

6.2.1 Aim

This part of Chapter 5 aims to help students navigate data management by first explaining what data and data management are, and why data sharing is important. It then provides advice and examples of best practices in data management.

This part runs over one class, and most of it is group work. I open with a single pass over the whole data life cycle, so that nobody leaves with a stage missing, and we spend the rest of the session on a project you have inherited from somebody who managed none of it.

💡 Two things to do before class

  1. Read the seven stages of the data life cycle. My pass over them in class is a synthesis, not your first contact with them. The list of sections, with links, is in the Before Class Activity.
  2. Set up a folder and a script on your computer, and download the project you are about to inherit. This is a short piece of hands-on work, not reading. Do not tidy that project up. Its state is the exercise.

PART A runs over two classes, and its reporting happens in a third session. Class 1 covers the life cycle and takes the activity as far as Step 1, and Class 2 builds the log and diagnoses the area each pair owns. The reporting is not a round of its own: it is Steps 3 to 5 of the PART B activity, where the data defects you logged are reported together with the code defects from the same project.

Before class

Block Time Who
The seven stages of the life cycle, read in advance 25 min Students, at home
Set up your folder and script, and download the project 10 min Students, at home

Class 1 — the life cycle, and a first look at what you inherited

Block Time Who
👥 Chapter 4: FAIR pairs report back 4 min Students, in 4 pairs
👥 Chapter 4: CARE Principles Activity 15 min Students, in 4 pairs
One Pass Over the Whole Cycle 15 min Instructor
five-minute break 5 min
👥 Step 1: Look at What You Have Been Given 5 min Whole class
Total in class 44 min

Class 2 — building the log, and diagnosing your area

Block Time Who
👥 Step 2: Open a csv and See What You Got Back 7 min Live-coded together
👥 Step 3: Build Your Defect Log 8 min Live-coded together
👥 Step 4: Diagnose the Area Your Pair Owns 20 min Students, in 4 pairs
Total 35 min

Reporting, in the PART B class

There is no third table here, because the reporting is not a block of its own. It is Steps 3 to 5 of Read the Code You Inherited, which open the second PART B class: your pair reports its data defect and its code defect in the same three minutes, writes a clause for each, and merges both into your charter.

6.2.2 Introduction

Good data management is essential for research excellence. It ensures that research data are of high quality, accessible to others, and usable in the future (see the TOP Guidelines in Chapter 3).

The value of data is increasingly recognized through formal citation. Repositories such as GBIF and Dryad provide DOIs that allow datasets to be cited, giving researchers credit for sharing their data. By making data available for reuse, researchers can contribute to their field while also enhancing their own professional visibility.

To ensure data citations are properly acknowledged, they must be included in publications—typically in a dedicated “Data Availability Statement” section at the end of the article. (See Figure 6.1 for an example.)

Excerpt of typeset journal text listing where each dataset lives: NCBI GenBank accession ranges for ITS and rbcL sequences, a BioProject and BioSample accession, SRA run numbers for raw sequence files, and a Zenodo DOI for the DNA alignments.

Figure 6.1: Example of a Data Availability Statement.

6.2.3 Learning Outcomes

By the end of this part, students are able to:

The fifth and sixth of these are assessed directly. Your individual project must include a data management workflow specific to your research, covering Create, Process, Document, Preserve, Share and Reuse. This part of Chapter 5 is where you acquire what you need to write it.

6.2.4 Before Class Activity

Do this before you come to class. You are going to inherit a badly managed research project, and the class has only seventy minutes in it. If the download happens in the room, we lose the first fifteen of them.

There is a second reason to do it this way. This part of the course is about managing data, and you are going to spend the class judging somebody who gathered theirs carelessly. Downloading a file by clicking a button leaves no record of where it came from. Doing it in a script does. So you will set up a small, tidy project of your own, and the first thing in it will be a script that says exactly where your data came from.

6.2.4.1 What to Read Before Class (25 minutes)

Read these seven sections, in this order. They are the seven stages of the data life cycle (Figure 6.2), and they all sit under Resources below. They are short, and the class assumes you have read them: my eighteen minutes in the room is a synthesis of these, not a substitute for them, and the group activity asks you to judge a real project against them.

  1. Planning Data Management — what has to be decided before any data exist, and why this is the stage that prevents everything else
  2. Creating Data — collection and digitisation, and the quality checks that belong at each step
  3. Processing Data — file formats, file naming, folder structure and version control
  4. Documenting Data — project-level and data-level documentation, plus Metadata
  5. Preserving Data — three copies, backup strategy, storage and format obsolescence
  6. Sharing Data — how and where data are shared, including Data Repositories and When Data Can’t Be Shared
  7. Reusing Data — rights, licences, credit and expectations

💡 Two short extras worth your time

  • Types of Research Data, because the group activity opens by asking which of the five types you are holding, and your answer changes how much everything else matters.
  • The Data Management Checklist. Do not read it through; just see what is in it. It is the tool you will reach for most during the activity, and knowing roughly where things are in it will save you time.

6.2.4.2 What to Set Up on Your Computer Before Class (10 minutes)

This part is hands-on, and it has to be done before you walk in. In five short steps you will make a folder for this chapter, start an R script in it, and use that script to download the project you are going to diagnose in class. None of it is difficult, but it is the difference between starting the activity at the beginning of the hour and starting it fifteen minutes in.

You end up with a small, tidy project of your own, containing one very untidy project belonging to somebody else. That contrast is the point, so work through all five steps rather than just grabbing the download.

6.2.4.2.1 Step 1: Create the Folders

Go to your Chapters folder, the one that already holds the Chapter_2 folder you built earlier in the semester. Inside it, create a single new folder named Chapter_5. Use Finder, File Explorer, or whatever you normally use.

Chapter_5 must sit beside Chapter_2, not inside it. When you are finished, that part of your course folder looks like this:

Chapters/
├── Chapter_2/   <- already there from earlier in the semester
└── Chapter_5/   <- new, and everything for this chapter goes in here

💡 Where is Chapters on my machine?

Wherever you put it when you set up Chapter_2. On the instructor’s computer the path is Course_Reproducible_Science/Chapters, so Chapter_5 ends up at Course_Reproducible_Science/Chapters/Chapter_5.

Yours does not have to match, and the part of the path above Chapters does not matter. What matters is that Chapter_5 is a sibling of Chapter_2, so that one folder holds your work for every chapter in the same shape. That is the consistent folder structure this chapter asks of you, applied to your own course materials: if you cannot keep five chapters tidy, a three-year project has no chance.

One folder serves the whole of Chapter 5, both PART A and PART B. You are about to download a project and spend PART A diagnosing its data; in PART B you come back to the same project and diagnose its code. There is no reason to hold two copies of it, and two copies is exactly the defect you are about to go and find in somebody else’s work.

Each part gets its own script in this folder, named for what it does.

6.2.4.2.2 Step 2: Create Your Script

Open RStudio, then File > New File > R Script, and save it immediately as:

`Chapter_5_data_management.R`

inside Chapter_5/. Save it before you type anything, because the next step depends on the file having a location on disk.

6.2.4.2.3 Step 3: Point R at That Folder

Your script now sits in the folder you want to work in, so tell R to work there too:

`Session > Set Working Directory > To Source File Location`

Then confirm it, and read what comes back rather than assuming:

# This must end in Chapter_5
getwd()

If the path does not end in Chapter_5, stop and fix it now. Everything below downloads into whatever directory this returns, which is the same lesson as Part D of Chapter 2.

6.2.4.2.4 Step 4: Download the Project, From Your Script

Type the code below into Chapter_5_data_management.R and run it. Put it in the script, not straight into the console: the script is the record, and a line of code in the console is gone the moment you close RStudio.

# Fetch the inherited project. mode = 'wb' matters: a zip
# is a binary file, and without it the archive arrives
# corrupted on some systems
download.file("https://github.com/svenbuerki/EEB603_Reproducible_Science/raw/master/Exercises/Chapter_5/Messy_Project.zip",
    destfile = "Messy_Project.zip", mode = "wb")
# Unpack it
unzip("Messy_Project.zip")
# Look at what you have just inherited
list.files("Messy_Project")
6.2.4.2.5 Step 5: Check You Have It

list.files() should return eight files:

[1] "analysis_USE THIS ONE.R"  "analysis.R"
[3] "analysis2.R"              "copy of data(1).csv"
[5] "Data final v2 FINAL.xlsx" "notes.txt"
[7] "plot data 12-03-25.csv"   "sites_landowners.csv"

Your own folder should now look like this:

Chapters/
├── Chapter_2/
└── Chapter_5/
    ├── Chapter_5_data_management.R   <- your script: tidy, and it says where the data came from
    ├── Messy_Project.zip
    └── Messy_Project/               <- somebody else's work: not tidy at all
        ├── analysis_USE THIS ONE.R
        ├── analysis.R
        ├── analysis2.R
        ├── copy of data(1).csv
        ├── Data final v2 FINAL.xlsx
        ├── notes.txt
        ├── plot data 12-03-25.csv
        └── sites_landowners.csv

💡 Do not tidy Messy_Project/

Do not rename anything in it, do not delete the duplicates, do not convert the spreadsheet. Its state is the exercise. Come to class with it exactly as it arrived, and bring your laptop: you will be working in these files in your pair.

💡 If the download fails

Open the file on GitHub and use the Download raw file ⇩ icon, the same way you fetched the image file in Chapter 2, then move the zip into Chapter_5 and unzip it by hand. Use this as a fallback only, and if you need it, write a comment in your script saying so. A script that admits a manual step is still honest; a script that silently omits one is not.

💡 Prefer to create the folders in R?

You can, and it is arguably the better habit, since it leaves the same record as the download does. Set your working directory to your Chapters folder, the one holding Chapter_2, then run this before saving the script and carry on from Step 2: 

#Check you are in Chapters/ before running the next line
getwd()

dir.create("Chapter_5")

If you ever need to create a folder inside a folder that does not exist yet, add recursive = TRUE. Without it R refuses, because it will not invent the parent for you.

✅ Checkpoint: you have Chapter_5/ holding your script, the zip, and an unzipped Messy_Project/ with eight files in it, and your script contains the download.file() and unzip() lines that produced them.

6.2.4.2.6 Step 6: Optional, Collect the Defect Log Templates

You will build Defect_log.csv and Defect_log.Rmd yourself in Step 3 of the activity, and building them is the point: it is the same work as every R Markdown document you have produced since Chapter 2. So do not start from the templates by default.

They exist for two situations: you want to check your own against a working copy, or the class is short of time and you need to get to the diagnosis. Both files are in the repository at Exercises/Chapter_5.

✅ Solution: download both templates

Download both files:

On each page use the Download raw file button, and save both into your Chapter_5 folder beside Messy_Project/.

Then knit Defect_log.Rmd once. It reads the csv, renders it with kable() and counts the rows with inline code, so a successful knit confirms the whole chain works before you have any findings to lose.

The .csv arrives with the header row and one worked example underneath it, which is both the format and the standard a row has to meet.

6.2.5 Class Outline

This session opens by finishing Chapter 4, so Chapter 5 itself starts about 20 minutes in.

  1. 👥 Chapter 4: FAIR pairs report back (4 minutes)
  2. 👥 Chapter 4: CARE Principles Group Activity (15 minutes)
  3. One Pass Over the Whole Cycle (15 minutes, instructor)
  4. five-minute break
  5. 👥 Group Activity: Diagnose an Inherited Project (40 minutes in four steps, split across two classes: today you do Step 1 only, 5 minutes; Steps 2 to 4 are the next class, and reporting is in the PART B class)

That is the whole of the class. Everything else in this part is reference material, gathered under Resources below and organised stage by stage, so that you can find what you need quickly while you are diagnosing. The Data Management Checklist at the end of it is the tool you will reach for most.

6.2.6 Resources

Everything you need to read for this part is in this section, as the subsections below. It is organised stage by stage, in the order of the data life cycle, so that you can find the one you need quickly while you are diagnosing a project. Your pre-reading list says which of them to read before class.

The whole part is based on the following sources:

6.2.6.1 Teaching Material

The presentation associated with this class is available here:

💡 Best browser to view presentation

  • For optimal viewing of this presentation in full-screen presentation mode, the instructor recommends using Mozilla Firefox. Its built-in PDF viewer supports seamless full-screen display with easy navigation tools, making it ideal for presentations without the need for external plugins. Other browsers may not handle PDF presentation features as reliably.

6.2.6.2 Types of Research Data and Their Specific Characteristics

Research data are the factual pieces of information used to test research hypotheses.
Data can be classified into five categories:

  • Observational: Data tied to a specific time and place that are irreplaceable (e.g., field observations, weather station readings, satellite data).
  • Experimental: Data generated in a controlled or partially controlled environment, which can be reproduced—though it may be expensive to do so (e.g., field plots, greenhouse experiments, chemical analyses).
  • Simulation: Data generated from models (e.g., climate or population modeling).
  • Derived: Data not collected directly but inferred from other data files (e.g., population biomass calculated from population density and average body size data).
  • Metadata: Data about data (more on this category later).

A key challenge facing researchers today is the need to work with diverse data sources. It is not uncommon for projects to integrate multiple data types into a single analysis—even drawing on data from disciplines outside of Ecology and Evolution. As research becomes increasingly collaborative and interdisciplinary, issues surrounding data management are becoming more prevalent.

6.2.6.3 The Data Life Cycle

Data have a longer lifespan than the project they were created for, as illustrated by the data life-cycle displayed in Figure 6.2.

Circular diagram of seven grey ellipses: a central Plan node sends arrows outward to every other stage, while a ring of double-headed arrows connects Create, Process, Document, Preserve, Share, and Reuse in sequence back around to Create.

Figure 6.2: The data lifecycle

Some projects may only focus on certain parts of the data life-cycle, such as primary data creation, or reusing others’ data. Other projects may go through several revolutions of the cycle. Either way, most researchers will work with data at all stages throughout their career.

6.2.6.3.1 One Pass Over the Whole Cycle (15 minutes)

This is the one part of this class I lecture, and it is the spine of everything that follows. We start from the five types of research data, because which type you hold decides how much a mistake costs you: an experimental dataset can be regenerated at a price, while an observational one is tied to a time and a place and is simply gone. Then we walk the cycle. Each row of the table below is a stage of Figure 6.2. For each one I give you what the stage is for, the failure it invites, and the practice that prevents that failure. The third column is what you are accountable for: it is where the clauses of your charter come from, and every defect in the project you inherited later in this class is a stage whose third column was ignored.

The Stage column links to the detailed treatment you read before class. Use those sections as your reference during the activity.

Stage What it is for The failure it invites What prevents it
Plan Deciding, before any data exist, who does what, in which formats, backed up where, and shared under which licence Every decision below gets made by accident, one file at a time A written plan, reviewed as the project changes, not a template filled in once
Create Collecting in field or lab and digitising into a raw dataset The one chance to record a measurement is taken badly, and no later work can repair it Calibrated instruments, standard templates and protocols, quality checks at each step, and context recorded as it happens
Process Turning the raw dataset into something analysable, without destroying the raw dataset The raw data are edited in place, so there is no longer anything to go back to Raw stays raw, open formats, one documented naming convention, and cleaning done in a script rather than by hand
Document Writing down what a reader needs in order to interpret the data without asking you The dataset becomes unreadable the moment you are unavailable, which includes your future self Project-level and data-level documentation, a data dictionary with units and codes, and metadata in a standard schema
Preserve Making sure the files still exist, and still open, years from now One laptop, one accident, and the project is gone; or the files survive in a format nothing can read Three copies in two places, open formats, integrity checks, and migration every few years
Share Putting the data where others can find, cite and reuse them Data that cannot be found, cited or credited, or sensitive data released that should never have left A trusted repository with a DOI, a licence, and an explicit decision about what must not be shared
Reuse Letting the data answer questions you never asked Reuse without credit in one direction, and unusable deposits in the other Clear rights and licences, a citable DOI, stated expectations for credit, and the same standards applied to data you reuse

💡 Plan sits in the middle of the figure for a reason

In Figure 6.2 Plan is not the first stage of the ring; it is the hub, with an arrow to every other stage. That is the single most useful thing in the diagram. Each of the six failures in the table above is cheap to prevent at the planning stage and expensive or impossible to repair later: you cannot re-measure an animal you released, you cannot recover a raw file you overwrote, and you cannot un-publish a landowner’s phone number.

So when you are asked to produce a data management plan, the question is not which template to fill in. It is: which of these six failures is my project most likely to suffer, and what am I doing now to make it impossible?

6.2.6.4 Benefits of Effective Data Management

Data management involves planning for all stages of the data life cycle and implementing this plan throughout your research project. When done effectively, it keeps the data life cycle moving smoothly, improves research efficiency, and ensures your data meet expectations set by yourself, funders, institutions, legislation, and publishers (e.g., copyright, data protection).

To put this into perspective, ask yourself:

Would a colleague be able to take over my project tomorrow if I disappeared, or make sense of the data without talking to me?

If your answer is YES, then you are managing your data well!

Effective data management brings several important benefits, including:

  • Ensuring data are accurate, complete, authentic, and reliable.
  • Increasing research efficiency.
  • Saving time and money in the long run—‘undoing’ mistakes is frustrating.
  • Meeting funder requirements.
  • Minimizing the risk of data loss.
  • Preventing duplication by others.
  • Facilitating data sharing.
6.2.6.4.1 Why Should I Share My Data?

Data sharing is increasingly required by funders and publishers. Many funding bodies, such as the NSF, now ask for data management and sharing plans as part of grant applications. Although sharing data can feel daunting, data are valuable resources whose usefulness often extends beyond their original purpose.

Benefits of sharing data include:

  • Increasing the impact and visibility of your research.
  • Encouraging collaborations and partnerships with other researchers.
  • Enhancing transparency and accountability.
  • Supporting the improvement and validation of research methods.
  • Reducing costs by avoiding duplication of data collection.
  • Advancing science by enabling others to use data in innovative ways.
6.2.6.4.2 When Data Can’t Be Shared

There are legitimate reasons why some data cannot be shared:

  • The datasets contain sensitive information about endangered or threatened species.
  • The data include personal information—sharing them could breach protocols or legal requirements.
  • Portions of the data are owned by others, and you do not have the rights to share them.

During your project’s planning stage, identify which data cannot or should not be shared. Journal data archiving policies acknowledge these valid exceptions.

6.2.6.5 Planning Data Management

Whether or not your funder requires a data management or sharing plan as part of your grant application, having a plan in place before starting your research project will prepare you for potential data management challenges (see the Data Management Checklist section below).

6.2.6.5.1 Before You Start Planning

Before designing your data management workflow, consider the following:

  • Check funder requirements for data management plans.
  • Consult your institution—especially regarding available resources and data policies.
  • Consider your project’s budget.
  • Talk to your supervisor, colleagues, and collaborators.
6.2.6.5.2 Key Considerations When Planning
  • Time: Creating a data management plan can take significant time. It’s not just a matter of filling out a template or checklist. A well-thought-out plan ensures that good data practices are embedded throughout the entire research process.
  • Tailored design: Your data management strategy should reflect the specific goals, methods, and context of your research project.
  • Roles and responsibilities: While one person may draft the plan, data management often involves multiple people at different stages. A strong plan supports coordinated workflows and clear communication among team members.
  • Review and adapt: Plan how you will review and update your data management approach throughout the project. Regular reviews help integrate best practices and catch problems early—before they grow into larger issues.
6.2.6.5.3 Creating Data

In the data life cycle (Figure 6.2), data creation begins when a researcher collects information in the field or lab and digitizes it to produce a raw dataset.

# Workflow associated with creating data

Collect data (in the field and/or lab) --> Digitize data --> Raw dataset

!! Perform quality checks @ each step to validate data !!

Quality control during data collection is essential, as there is often only one opportunity to gather data from a given situation. Researchers should critically evaluate their methods before data collection begins—high-quality methods lead to high-quality data. Once collection is underway, it’s equally important to document the process in detail as evidence of data quality.

6.2.6.5.3.1 Key Things to Consider During Data Collection
  • Logistical challenges in the field: What potential issues might arise during fieldwork? For example, is there no access to power for charging equipment or devices?
  • Calibration of instruments: Are the tools you’re using—e.g., balances or sensors—accurate and suitable for your research needs?
  • Multiple measurements or samples: Consider taking replicate observations or samples to improve statistical robustness or capture genetic variation within a population.
  • Standardized templates and protocols: Create templates and clear protocols to ensure consistent data collection—especially important when multiple people are collecting data.
  • Environmental or contextual factors: Document any conditions that could influence data quality, such as extreme weather events affecting wildlife observations.
  • Metadata from data collectors: Use a short questionnaire for all data collectors to capture contextual variables or collection practices that might affect the analysis. This helps explain variability during the analytical phase.
  • Adding relevant variables: Enhance the future usability and impact of your data by including additional variables (e.g., landscape features or environmental parameters) that may open up new research avenues.
6.2.6.5.4 Data Digitization

Data may be collected directly in digital form using electronic devices, or recorded manually as handwritten notes. In either case, some processing is required to produce a structured, digital raw dataset.

Key things to consider during data digitization include:

  • Database structure: Design your data structure early. Ensure observations have unique identifiers and data files are clearly organized.
  • Consistent data format: Use a standardized structure—e.g., one row per record, with columns representing variables (a common “spreadsheet” format).
  • Atomization of data: Ensure each cell contains only one data point. This greatly improves the ease and accuracy of analysis.
  • Plain text encoding: Use plain characters (e.g., ASCII) to maximize software compatibility and prevent formatting issues.
  • Coding variables: Assign numerical codes to categorical variables to support statistical analysis. Keep coding systems simple and consistent.
  • Metadata and documentation: Describe your dataset thoroughly in a Readme.txt or protocol file (ideally attached as an appendix to your manuscript). Include definitions for all variables, units of measurement, and codes for missing values.
  • Preserve raw data: Always keep an untouched copy of the original dataset—raw data should remain raw.
6.2.6.5.5 Processing Data

Data should be processed into a format that supports analysis and ensures long-term usability. Data are at risk of being lost if the hardware or software used to create or process them becomes obsolete. To prevent this, data should be well organized, clearly structured, appropriately named, and version-controlled using standard, future-proof formats (see the Data Structure and Organisation of Files section below).

Here are some best practices for data processing (Figure 6.2):

  • File formats: Use non-proprietary (open standard) formats whenever possible. These formats are accessible without specialized software, ensuring long-term usability. For example:
    • .csv for tabular data
    • .txt for plain text
    • .gif, .jpeg, or .png for images
      Avoid formats tied to specific software unless necessary.
  • File names and folder structure: Maintain a clear and consistent folder hierarchy that mirrors the structure of your project. Organize data logically and label files descriptively so they are easy to locate.
    • Creating a “folder map” (stored in the Readme.txt file) can help others navigate your project or aid your own memory over time.
  • Good file naming practices:
    File names should be:
    • Unique
    • Descriptive
    • Concise
    • Consistently structured
    • Include relevant metadata such as project name, content, location, date, researcher initials, and version number
      > ⚠️ Avoid using spaces in file names — they can cause issues in scripts and metadata processing.
  • Quality assurance: Before analysis, ensure your data are clean, consistent, and reliable by editing, verifying, and validating the dataset. Use a scripting language like R to process and check your data, so all steps are transparent and reproducible.
    Quality assurance checks may include:
    • Identifying missing values, estimated values, or duplicate entries
    • Running summary statistics to detect outliers or implausible values
    • Verifying data formats and consistency across fields
    • Comparing against reference datasets to detect discrepancies
  • Version control: Tracking versions of your data files is critical, especially when working with collaborators or across multiple locations. A strong version control strategy helps manage changes, avoid confusion, and maintain data integrity.
    Version control best practices include:
    • Defining which versions to keep and for how long
    • Using a systematic naming convention (e.g., v1_draft, v2_reviewed, v3_final)
    • Recording changes in a separate changelog or version table
    • Mapping version locations if files are stored across systems
    • Synchronizing versions between collaborators or storage systems
    • Ensuring that any files referencing one another are updated together
6.2.6.5.6 Data Structure and Organisation of Files

A clearly structured directory tree offers a simple yet powerful way to summarize your project’s file organization. It improves clarity, supports reproducibility, and makes onboarding easier for collaborators.

You can produce one without leaving R, and that is the route to learn first, because it behaves the same on macOS, Windows and Linux. The example below inspects Project_ID, the worked example that ships with the course repository:

#Every file in the project, as a path relative to the project root
list.files("Project_ID", recursive = TRUE)
##  [1] "01_Raw_Data/species_data.csv"                   
##  [2] "02_Processed_Data/species_data_clean.csv"       
##  [3] "03_Scripts/01_download_data.R"                  
##  [4] "03_Scripts/02_clean_data.R"                     
##  [5] "03_Scripts/03_exploratory_analyses.R"           
##  [6] "03_Scripts/04_fit_models.R"                     
##  [7] "03_Scripts/05_generate_figures.R"               
##  [8] "04_Output/DataStr_network.html"                 
##  [9] "04_Output/DataStr_tree.pdf"                     
## [10] "05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf"
## [11] "06_Reports/Documentation.md"                    
## [12] "07_MS/MS_et_al.docx"                            
## [13] "LICENSE"                                        
## [14] "R_functions/check.install.pkg.R"                
## [15] "README.md"

Read the paths and you have read the project. Each one tells you what the file is from where it sits, and the leading numbers say in which order the stages run. For the directories alone:

#Just the folders. The empty string is the project root itself
list.dirs("Project_ID", recursive = TRUE, full.names = FALSE)
## [1] ""                      "01_Raw_Data"           "02_Processed_Data"    
## [4] "03_Scripts"            "04_Output"             "05_Figures_and_Tables"
## [7] "06_Reports"            "07_MS"                 "R_functions"

If you would rather see it drawn as a tree, the data.tree package prints one. This is the output the rest of this chapter refers to, and it is generated from the real folder each time this page is built, so it cannot fall out of step with the example:

#Combine the file list with its metadata, then let data.tree draw the structure
filesInfo <- file.info(list.files("Project_ID", recursive = TRUE, full.name = TRUE))
data.tree::as.Node(data.frame(pathString = rownames(filesInfo)))
##                            levelName
## 1  Project_ID                       
## 2   ¦--01_Raw_Data                  
## 3   ¦   °--species_data.csv         
## 4   ¦--02_Processed_Data            
## 5   ¦   °--species_data_clean.csv   
## 6   ¦--03_Scripts                   
## 7   ¦   ¦--01_download_data.R       
## 8   ¦   ¦--02_clean_data.R          
## 9   ¦   ¦--03_exploratory_analyses.R
## 10  ¦   ¦--04_fit_models.R          
## 11  ¦   °--05_generate_figures.R    
## 12  ¦--04_Output                    
## 13  ¦   ¦--DataStr_network.html     
## 14  ¦   °--DataStr_tree.pdf         
## 15  ¦--05_Figures_and_Tables        
## 16  ¦   °--Fig_01_Data_lifecycle.pdf
## 17  ¦--06_Reports                   
## 18  ¦   °--Documentation.md         
## 19  ¦--07_MS                        
## 20  ¦   °--MS_et_al.docx            
## 21  ¦--LICENSE                      
## 22  ¦--R_functions                  
## 23  ¦   °--check.install.pkg.R      
## 24  °--README.md

💡 Why generate this rather than paste in a screenshot

An earlier version of this chapter showed a picture of a terminal here. When the example project was reorganised, the picture silently became a lie: it described folders that no longer existed, in a chapter about keeping documentation in step with reality. The three blocks above are produced by running the code against the folder, so that cannot happen again.

The same reasoning applies to your own work. A figure you generate is documentation; a figure you paste is a claim about the past.

6.2.6.5.6.1 The UNIX tree Package: Supporting Data Structure and Organisation

In a terminal, on macOS or Linux, the tree package does the same job in one command, and prints a more compact picture than R does. It is worth having if you work at the command line, but it is an extra, not a requirement: everything above already works on any machine with R.

The UNIX tree package can be used to generate a visual representation of your project’s directory structure. Run on the Project_ID/ folder it is a helpful way to check file names, file locations, and version control practices.

Some useful tree command options include:

  • -s — Displays the size (in bytes) of each file alongside its name.
  • -D — Shows the date and time of the last modification for each file.
    > Tip: If used with the -c option, it instead shows the time of the last status change.

These options make it easy to identify inconsistencies in file naming, outdated versions, or misplaced files.

To install the tree package on macOS, follow these steps:

#Install tree by typing the following code in a terminal
brew install tree
6.2.6.5.7 Documenting Data

Creating clear documentation and metadata is essential for ensuring that your data can be understood, interpreted, and reused over the long term (Figure 6.2). Documentation—essentially the manual accompanying your dataset—should describe the data, explain any transformations or processing steps, and provide contextual information. The goal is to leave no room for misinterpretation.

Documentation requirements should be identified during the planning phase, so they can be implemented throughout all stages of the data life cycle, especially during data creation and processing. This proactive approach helps avoid “rescue missions” later—such as trying to remember undocumented steps or dealing with missing information when a collaborator leaves the project.

Data documentation includes information at both the project and data levels.

6.2.6.5.7.1 Project-Level Documentation

This includes broader information about the research project and how data were collected and managed:

  • Project aims, objectives, and hypotheses
  • Personnel involved, including contact information for key contributors
  • Funding sources or sponsors
  • Data collection methods, including:
    • Instruments used
    • Environmental or field conditions
    • Copies of any data collection protocols
  • Standards and best practices followed
  • File structure and organization (see Data Structure and Organisation of Files)
  • Software used to create, process, or access the data
  • Processing procedures, including:
    • Quality control steps
    • Version control strategy
    • Dates when changes occurred
  • Known limitations or issues that might affect data interpretation
  • How to cite the dataset
  • Intellectual property rights, licenses, or restrictions
6.2.6.5.7.2 Data-Level Documentation

This focuses on the details within individual datasets:

  • Names, labels, and descriptions of variables
  • Explanations of coding schemes (e.g. numeric codes for categorical variables)
  • Definitions for acronyms or technical terms
  • Reasons for missing values or data gaps
  • Documentation of derived data, including:
    • Formulas or algorithms used
    • Code used to generate them (especially if processed using software like R)

If you are using a language like R or Python for processing, much of this data-level documentation can be embedded directly in your scripts or notebooks.

6.2.6.5.8 Metadata

Metadata support data discovery, reuse, and interoperability—especially in online repositories and databases. They also facilitate machine-readable access, which is increasingly important in large-scale, automated research workflows.

Metadata are typically created using: - A repository’s data submission form
- A metadata editor
- A standalone metadata generation tool (many of which are freely available online)

Metadata follow standard schemas and usually fall into three main categories:

  • Descriptive metadata — Includes the dataset’s title, author(s), abstract, keywords, etc.
  • Administrative metadata — Covers rights, permissions, file formats, and technical access details.
  • Structural metadata — Explains how data are organized, such as table relationships or file hierarchies.
6.2.6.5.9 Preserving Data

To protect against data loss and ensure long-term accessibility, every research project should include a clear strategy for data backup and storage (see where this step fits in Figure 6.2). A general recommendation is to maintain at least three copies of your data: - the original file, - an external/local backup (e.g. external hard drive), and - an external/remote backup (e.g. cloud storage or network drive).

Consult with your thesis supervisor or institutional IT team to determine the most suitable procedures for your project and research environment.

6.2.6.5.9.1 Data Backup

When designing a backup strategy, consider the various ways in which data could be lost or compromised, including:

  • Hardware failure
  • Software bugs or crashes
  • Virus infection or hacking
  • Power outages
  • Human error
  • Theft or misplacement (especially common during fieldwork)
  • Physical damage (e.g. fire or flooding)
  • Accidental overwriting of valid backups with corrupted data (this is more common than you might think)

While it’s not always possible to protect against every risk, you should tailor your backup strategy to address the most likely scenarios for your project and context.

Key considerations for a backup plan:

  • Which files require backup (e.g. raw data, scripts, figures, protocols)
  • Who is responsible for backing up the files
  • How frequently backups should be performed (depends on how often files change)
  • Whether to use full or incremental backups:
    • Incremental backups capture only recent changes and can be done frequently
    • Full backups create a complete snapshot and should be performed periodically
  • Backup procedures for each storage location (e.g. field devices, personal computers, servers)
  • How backup files are named, dated, and organized
6.2.6.5.9.2 Data Storage

Robust data storage is essential for both original and backup files. While this applies to paper records as well, electronic data present unique challenges related to format obsolescence, media degradation, and digital security.

Best practices for electronic data storage:

  • Use high-quality and reliable storage media
  • Save files in non-proprietary formats to ensure long-term readability
  • Migrate data every 2–5 years to newer storage media (e.g. avoid long-term use of CDs/DVDs or outdated drives)
  • Perform regular data integrity checks to ensure files have not been corrupted
  • Store copies in multiple formats and locations (e.g. external hard drive + cloud storage)
  • Label files and folders clearly with logical and consistent naming conventions
  • Encrypt sensitive data, especially when stored on portable devices or external drives
    > Password-protected computers may not protect data once it’s backed up externally

Where can data be stored and backed up?

  • Network Drives
    Managed by institutional IT staff, these are regularly backed up, secure, and typically access-controlled.

  • Personal Devices (e.g. laptops, tablets)
    Convenient for working, but not recommended for long-term or master copies due to higher risks of loss, damage, or theft.

  • External Devices (e.g. USBs, hard drives, CDs)
    Affordable and portable, but also vulnerable. Use only high-quality devices from reputable brands, and avoid relying on them for long-term preservation.

  • Remote or Cloud Services (e.g. Google Drive, Dropbox)
    Useful for syncing and off-site backup. Free plans are often limited, and advanced features may require a subscription. Check institutional policies before using third-party services for sensitive data.

  • Paper Copies
    Surprisingly resilient, paper copies of small datasets or key documents can complement digital backups. While not ideal for all data types, ink on paper offers long-term accessibility—as long as it’s stored safely and can be found when needed!

6.2.6.5.10 Sharing Data

Research data can be shared in a variety of ways, each with its own advantages and limitations. Choosing the appropriate method depends on the nature of your data, institutional and funder policies, and your goals for dissemination and reuse.

Common data sharing methods include:

  • Submitting data to a disciplinary repository (e.g. Dryad, GenBank)
  • Depositing data in your funder’s data center
  • Uploading data to your institution’s repository
  • Sharing via open notebooks or project websites (e.g. the Open Science Framework by the Center for Open Science)
  • Using virtual research environments (e.g. SharePoint, Sakai)

Before deciding on a sharing method, check the data sharing policies of your institution, funder, and target journals. These often include specific requirements for timing, formatting, licensing, and metadata.

This section focuses on the benefits of using data repositories as a preferred strategy for long-term sharing and preservation.

6.2.6.5.10.1 Data Repositories

Archiving your data in a trusted repository is one of the most effective and sustainable ways to share your research data. Repositories do require you to meet certain submission standards (data formatting, metadata completeness, and sometimes a minimum level of documentation), but meeting those standards is the service, not the cost. They are the same standards that make your data usable by you in five years.

What a trusted repository gives you that a project website cannot:

  • A persistent identifier. A DOI that resolves to your dataset indefinitely, so it can be cited in its own right (Figure 6.1) and counted as a research output.
  • Curation and preservation. Someone else’s institutional commitment to migrating formats and checking integrity, which is the part of Preserving Data you cannot realistically do alone for decades.
  • Standard metadata. A submission form built on a recognised schema, which is usually the fastest way to produce the metadata you need anyway.
  • Discoverability. Indexing, so that researchers who have never heard of you can find the data.
  • Access control. The ability to embargo a dataset until publication, or to publish the metadata openly while the data themselves are released only on request. That is how most of the cases in When Data Can’t Be Shared are handled in practice. Restricted is not the same as unshared: a record that says the data exist and how to ask for them is far better than silence.

Where to deposit. Prefer a disciplinary repository where one exists, because that is where people in your field look and the metadata standards are already tailored to the data type:

  • Sequence data: NCBI GenBank, the SRA, or the ENA.
  • Occurrence and biodiversity records: GBIF.
  • Anything without an obvious disciplinary home: a generalist repository such as Dryad or Zenodo, both of which issue DOIs and both of which are accepted by most journals.
  • Institutional repositories, which your library can advise on, and which may be required by your funder.

Choosing one is a planning decision, not a submission-day decision. Before you have data, check: does my target journal or funder mandate a particular repository? Does it issue a DOI? Does it support embargo or restricted access, if I will need that? Will it accept my file formats and my file sizes? A repository chosen after the analysis is finished is how people end up reformatting everything in the week a paper is accepted.

⚠️ Depositing is not the last step. The dataset’s DOI has to appear in the paper, in a Data Availability Statement (Figure 6.1), or nobody will ever follow it.

6.2.6.5.11 Reusing Data

All aspects of data management ultimately support the discovery and reuse of data by others. This is one of the main goals of reproducible science and open research.

To facilitate responsible reuse, it is essential to clearly document intellectual property rights, licenses, and permissions. These should be included in your data documentation and metadata so that others understand how your data can be reused.

At this stage of the data life-cycle, it is important to clearly communicate your expectations for reuse, which may include:

  • Acknowledgment in publications
  • Proper citation of the dataset (e.g. using the DOI)
  • Requests for co-authorship where substantial contributions are expected

Likewise, those who wish to reuse data must take responsibility for ethical and transparent use. This includes:

  • Giving appropriate credit to the original data creators
  • Citing the dataset properly
  • Following licensing terms and usage conditions
  • Managing any new research data with the same care and standards

When requesting access to someone else’s data, be transparent about your intentions. Clearly state: - The purpose of your request
- The research question or idea being explored
- Your expectations regarding collaboration or authorship

Co-authorship can be a sensitive and complex issue. It should be discussed openly with collaborators at the outset of a project to avoid misunderstandings later.

Openness to data reuse, combined with long-term data preservation, fosters collaboration, transparency, and innovation. Good data management enables high-quality datasets to remain useful beyond the original study and to contribute meaningfully to the broader research community—helping to answer the big questions in fields like ecology and evolution.

By making your data discoverable and reusable, you help ensure that science continues to grow on a strong, transparent foundation for future generations.

6.2.6.6 Data Management Checklist

The following checklist, adapted from the UK Data Archive, is designed to guide you in planning effective data management and sharing. Use it to help design your own data management plan and ensure best practices throughout your research.

6.2.6.6.1 Planning
  • Who is responsible for each aspect of data management?
  • Are there new skills or training needed to carry out data management activities?
  • Do you need additional resources such as personnel, time, or hardware to manage data effectively?
  • Have you accounted for costs related to depositing data for long-term preservation and access?
6.2.6.6.2 Documenting
  • Will others be able to understand and properly use your data?
  • Are your data self-explanatory in terms of variable names, codes, and abbreviations?
  • What descriptions and contextual documentation will explain the meaning, collection methods, and processing of your data?
  • How will you label and organize data, records, and files?
  • Will your cataloging be consistent and clear?
6.2.6.6.3 Formatting
  • Are you using standardized and consistent procedures to collect, process, transcribe, check, validate, and verify your data? (e.g., standard protocols, templates, or input forms)
  • What data formats will you use? Do these formats and associated software enable easy sharing and long-term sustainability (preferably non-proprietary or open-standard formats)?
  • When converting data across formats, do you verify that no data, annotations, or internal metadata have been lost or altered?
6.2.6.6.4 Storing
  • Are your digital and non-digital data—and any copies—stored securely in multiple safe locations?
  • Do you need to securely store personal or sensitive data? If so, are appropriate protections in place?
  • How will data collected on mobile devices be transferred and stored?
  • If data are held in multiple places, how will you track different versions?
  • Are your files backed up regularly and stored safely?
  • Do you know which version of your data files is the master copy?
  • Who has access to your data during and after the research? Are access restrictions needed, and how will they be managed long-term?
  • How long will you keep your data? Have you decided which data to preserve or destroy?
6.2.6.6.7 Sharing
  • Do you plan to share all your data, or will you select which data to preserve and share?
  • How and where will you preserve your research data for the long term?
  • How will you make your data accessible to future users?

6.2.7 👥 Group Activity: Diagnose an Inherited Project (40 minutes, split across two classes)

6.2.7.1 Aim

You have inherited a research project, and your job is to review it against the data management practices set out in this part. The project is Messy_Project, the folder you downloaded before class. It belonged to a student who has left the lab: the data are observations that cost a field season, there is a manuscript half written, and the person who knows how any of it works is not answering email. Everything you need is in that folder, or it is lost.

This is the question from Benefits of Effective Data Management turned around on you:

Would a colleague be able to take over my project tomorrow if I disappeared, or make sense of the data without talking to me?

Today you are that colleague, and the answer is no. Your task has two halves:

  1. Identify the defects. A defect here is any property of the project that stops somebody other than its author from finding, reading, trusting or reusing the work: a file whose name does not say which version it is, a measurement stored with its unit inside the value, a variable nobody has defined, a dataset with no statement of who may see it. A defect is not a matter of taste. It is a place where the project departs from a practice this part sets out, which is why every finding has to name the practice it breaks.
  2. Say what would fix it, and what cannot be fixed. For each defect you write a mitigation: the repair, in pseudocode, where repair is still possible; and where it is not, a plain statement of what has been permanently lost and what would have had to happen earlier to prevent the loss. That second kind is the more valuable finding, because it is the one that changes how your own group works rather than how this folder looks.

The output is a written record, your Defect_log, which becomes the evidence for the standards you add to your group’s reproducibility charter.

💡 This project is fictional, and every defect in it is deliberate

The measurements, the site coordinates and the names in the folder are invented for this exercise. Nothing describes a real person or a real population. But none of the defects are exaggerated. Every one of them is something your instructor has been handed in a real project.

6.2.7.2 Learning Outcomes

By the end of this activity, students are able to:

6.2.7.3 Requirements

Two things have to be on your laptop before the activity starts. Both were set up in What to Set Up on Your Computer Before Class.

⚠️ You cannot do this activity without the two things you set up before class

Everything below works on a folder that has to be already on your laptop. If it is not, you will spend the activity downloading instead of diagnosing. You need:

  • The project itself, unzipped: Chapter_5/Messy_Project/, downloaded with your own script in What to Set Up on Your Computer Before Class. Leave it exactly as it came: its mess is the exercise.
  • Your script, Chapter_5_data_management.R, open in RStudio. Every command today goes in it, so that at the end of the session your own work is reproducible even though the project you are diagnosing is not.

If either is missing, fix it now, in the first two minutes, following the five setup steps. Do not start reading the folder until both are in place, and do not borrow a neighbour’s copy: you need it on your own machine for Step 2.

6.2.7.4 Overview

Seven steps over three classes. You leave the second class with a written diagnosis, not a finished activity.

Step Who Time When
1 Look at what you have been given everyone 5 min Class 1
2 Open a csv and see what you got back together 7 min Class 2
3 Build your Defect_log together 8 min Class 2
4 Diagnose the area your pair owns your pair 20 min Class 2
5 to 7 Report, write the clause, merge into your charter your pair, then your group 45 min, with PART B’s the PART B class

Classes 1 and 2 are both this part’s own sessions. There is no Class 3: the reporting is Steps 3 to 5 of the PART B activity, where your data defects are reported alongside the code defects from the same project.

Reference, to look up when you need it rather than to read in order:

💡 Today you diagnose. You do not repair

Bad data rarely announces itself: it loads without error, it analyses without complaint, and it produces a number that looks fine. Everything downstream depends on somebody noticing first, which is why detection is the skill being practised today.

So keep the folder exactly as it is. People start renaming files within a minute of opening a messy folder, before they understand what they are looking at, and destroy the evidence doing it. Repairing data properly also needs R skills we come to later, which is why your fixes are written as pseudocode rather than run.

One more reason the activity spans two classes. A clause you write today is a promise: we will keep the raw data untouched. In the PART B class you read the same project’s code, which is what turns a promise into something somebody can actually check.

6.2.7.4.1 Your pair, and the kind of fault it owns

Work in four pairs. This project is broken in four different kinds of way, and each pair takes one kind. We call the kind you take your failure class: not a single mistake, but a family of related ones, such as everything to do with how the files are named and arranged, or everything to do with what is written down about them.

Splitting the work this way is deliberate. Left to itself, a whole class opening the same messy folder finds the same three obvious problems and stops. Giving each pair one kind forces somebody to look hard at the parts nobody would have chosen to look at, which is where the expensive faults usually are.

The sections in the third column are the ones you read before class: they are your reference, and they are also the standard you are judging the project against. When you ask whether something is a defect, the answer is not your opinion; it is whether the project does what those sections say it should.

👥 Form the pairs inside your charter groups

You have two charters from Chapter 1 and there are four pairs, so split each charter group into two pairs rather than pairing across groups:

  • Charter group 1 → Pair 1 and Pair 2
  • Charter group 2 → Pair 3 and Pair 4

This matters for Step 5. When you come to write the findings into your charter, you will be sitting with the other pair from your own group, so two of the four clauses are already in the room and the other two arrive in writing.

Pair Owns the failures of Your reference sections
Pair 1 Naming and organisation: what the files are called and where they sit Processing Data, Data Structure and Organisation of Files
Pair 2 Formats and atomisation: what is inside the files Data Digitization, Processing Data
Pair 3 Documentation and metadata: what a reader would need in order to interpret any of it Documenting Data, Metadata
Pair 4 Preservation and sharing: whether the data will survive, and whether they may leave the lab Preserving Data, Sharing Data, Data Repositories, When Data Can’t Be Shared

Nobody owns Planning. That is not an oversight, and we will come back to it in Step 3.

6.2.7.5 Step-by-Step Procedure

Steps 1 to 3 are the same for everybody and set the activity up: you look at what you inherited, you learn what R does to a file when it opens it, and you build the log you will record findings in. We do Steps 2 and 3 together, with me writing the code at the front and you writing the same lines in your own script, so that nobody is held up by how much R they have met before. Step 4 is your pair’s own work. The timings are in the Overview above.

6.2.7.5.1 Step 1: Look at What You Have Been Given (everyone, 5 minutes)

Three moves, and you open no files in any of them.

  1. List what you inherited — one line of R returns the whole folder
  2. Read the listing — what an extension, a name and a size can and cannot tell you
  3. Answer the three questions — as a class, with your evidence every time
6.2.7.5.1.1 List What You Inherited

Before anyone opens a file, get a view of the whole thing: every file, how big it is, and when it was last touched. Do this in R, in the script you created before class, and keep working in that script for the rest of the session.

Add these lines to Chapter_5_data_management.R, below the download code, and run them:

# What did we inherit? Keep two of the columns file.info()
# returns: size = the file's size, in bytes mtime =
# modification time, when the file was last changed
file.info(list.files("Messy_Project", full.names = TRUE))[, c("size",
    "mtime")]

mtime is R’s name for modification time, the moment the file was last written to. file.info() returns several more columns; [, c("size", "mtime")] keeps just these two.

Why in R rather than by clicking around a folder? Three reasons, and they are the reasons this chapter exists. It behaves identically on macOS, Windows and Linux, so nobody is held up by what is or is not installed. It puts the same answer in front of all eight of you, in the same form. And it leaves a record: by the end of this class your script will say what you inherited, what you found wrong with it, and what you did about it. The student whose project you are holding kept no such record, which is most of why you are in this situation.

This is what comes back:

                                       size               mtime
Messy_Project/analysis_USE THIS ONE.R   347 2026-10-09 10:37:36
Messy_Project/analysis.R                 57 2026-10-09 10:37:36
Messy_Project/analysis2.R               156 2026-10-09 10:37:36
Messy_Project/copy of data(1).csv       626 2026-10-09 10:37:36
Messy_Project/Data final v2 FINAL.xlsx 7109 2026-10-09 10:37:36
Messy_Project/notes.txt                 343 2026-10-09 10:37:36
Messy_Project/plot data 12-03-25.csv    345 2026-10-09 10:37:36
Messy_Project/sites_landowners.csv      331 2026-10-09 10:37:36

💡 Read both columns, and then read what is missing

size: eight files, nine kilobytes between them. Ask which files are suspiciously close in size to one another, and which one is far larger than everything else, and why that would be.

mtime: every timestamp is identical, and your own will differ from the one above, because it is the moment you unzipped the archive, not the moment any of this work was done. That is worth sitting with. Packaging a project up destroyed the only remaining trace of what happened when, and no amount of care afterwards brings it back. A project’s history has to live in a written record, because file metadata will not keep it for you.

💡 You can also just look at it

Nothing stops you, and it is often quicker for getting the feel of an unfamiliar folder. In Finder use View > as List and widen the Size column; in File Explorer use View > Details; the Files pane in RStudio, bottom right, already shows one. On macOS or Linux, tree -s Messy_Project in a terminal gives you the whole folder in one screen, if you already have tree (there is an installation note earlier in this chapter, but do not spend class time on it).

Use these alongside the R output, not instead of it. Only the script remembers what you did.

6.2.7.5.1.2 Read the Listing

You can get a long way before opening anything, if you read the listing deliberately. You have three kinds of evidence and nothing else, and it is worth being clear about what each one can and cannot tell you:

What you read What it tells you What it cannot tell you
The extension (.csv, .xlsx, .R, .txt) what kind of thing this is: a table, a script, prose whether the contents really are that. An extension is a claim by whoever named the file, not a fact
The name what the author thought the file was, and usually whether they believed there were several versions of it (copy of, v2 FINAL, a date, a number) which of those versions anyone should use
The size which files plausibly hold the same content, because near-identical sizes rarely happen by accident, and which file is unlike all the others anything at all about whether the contents are correct

A dataset is one set of observations, however many files it has been saved into. Three files holding the same twelve measurements are one dataset in three versions, not three datasets. That distinction is the whole of the first question.

Apply it to this listing. Four files could hold tabular data: Data final v2 FINAL.xlsx, copy of data(1).csv, plot data 12-03-25.csv and sites_landowners.csv. The first three are named as though they concern the same work, and their sizes are not far apart once you allow for the .xlsx carrying a spreadsheet’s worth of overhead. You cannot yet tell whether that makes them three datasets or one dataset saved three times, and the listing will never tell you.

Why it matters more than tidiness. If they are versions of one another, then this project contains more than one recorded value for the same measurement, and nothing in the folder says which one is right. A name like v2 FINAL is a claim about which is newest, not about which is correct, and copy of says only that somebody copied something. Hold that question open: you will open one of these files in Step 2, and whoever owns formats and naming will be the ones to settle it.

6.2.7.5.1.3 Answer the Three Questions

Answer these as a class, from the listing alone, and say what your evidence is every time. An answer without its evidence is a guess, and part of the point of this exercise is learning to tell the two apart.

  1. How many datasets are in this folder? Not how many data files: how many datasets. Work it out in two moves: first, which files could hold tabular data at all, judging by extension? Then, of those, which look like versions of one another rather than different things, judging by name and size? State what made you group them the way you did.
  2. Which file is the master copy? Name one, and then say precisely what in the listing justifies naming it. If nothing does, say so, and identify the one piece of information that would have settled it instantly. That missing thing is the finding.
  3. Which script produced the numbers in the manuscript? Three files end in .R. Give your answer and your evidence, then interrogate the evidence: is it actually evidence, or is it just a filename telling you what to think? Being told which file to use is not the same as knowing.

💡 Two of these three questions cannot be answered

That is deliberate, and it is not a trick. You are expected to reach “I cannot tell, and here is exactly what I would need”, because that is the real experience of inheriting somebody’s project, and because naming the missing evidence is what you will turn into a charter clause in Step 4. A pair that answers confidently here has usually stopped reading the listing and started guessing.

6.2.7.5.2 Step 2: Open a csv in R and See What You Got Back (together, 7 minutes)

Everyone needs this, whatever your failure class, because all four sections ask you to open at least one of these files. We use base R only: nothing to install, nothing to load.

Read the file and keep it. Add this to your script:

# The path is relative to your working directory, which
# should be Chapter_5 Name the file once. The dictionary in
# Step 3 is named from this, so there is one place to
# change if you point at another file.
datafile <- "Messy_Project/copy of data(1).csv"
d <- read.csv(datafile)

Look at the file name you just had to type: copy of data(1).csv, with spaces and brackets in it. It works here only because it is inside quotation marks. That is Pair 1’s subject, so if that is you, start noticing names now.

What you get back is a data.frame, which is R’s name for a table:

class(d)  #what kind of object is this?
dim(d)  #how many rows, how many columns?
names(d)  #what are the columns called?
head(d, 3)  #show me the first three rows
> class(d)
[1] "data.frame"

> dim(d)
[1] 12  7

> names(d)
[1] "V1"            "x2"            "temp"          "trt"
[5] "mass"          "biomass_index" "notes"

> head(d, 3)
  V1      x2 temp trt   mass biomass_index     notes
1  1 3/12/25 25°C   1 12.5 g          0.84        ok
2  2 3/12/25   25   1   11,8          0.79
3  3 3/12/25   25   2   12.5          0.84 scale odd

So: 12 observations of 7 variables. In a data.frame, rows are observations and columns are variables, and you reach a single column with the dollar sign, as in d followed by $mass.

The one rule to carry with you: each column is a vector, and a vector holds exactly one type of thing. Different columns may hold different types, but within a column everything is the same type. Everything below follows from that.

read.csv() has to guess each column’s type, and one bad value changes its answer for the whole column. Ask R what it decided:

# What type did each column end up as?
sapply(d, class)
> sapply(d, class)
           V1            x2          temp           trt          mass
    "integer"   "character"   "character"     "integer"   "character"
biomass_index         notes
  "character"   "character"

Look at mass. It is a set of measurements, so it should be numeric, and R has made it character (text). It did that because the column is a vector and must hold one type: the moment R met a value it could not read as a number, the only type that could hold everything in that column was text. One unusable value is enough to turn a column of measurements into a column of words, and the same has happened to temp and to biomass_index.

str() shows all of this in one go, and is the single most useful function here:

# Structure: dimensions, column names, types and first few
# values, all at once
str(d)
> str(d)
'data.frame':   12 obs. of  7 variables:
 $ V1           : int  1 2 3 4 5 6 7 8 9 10 ...
 $ x2           : chr  "3/12/25" "3/12/25" "3/12/25" "12.03.2025" ...
 $ temp         : chr  "25°C" "25" "25" "24.5" ...
 $ trt          : int  1 1 2 2 3 3 1 1 2 2 ...
 $ mass         : chr  "12.5 g" "11,8" "12.5" "-999" ...
 $ biomass_index: chr  "0.84" "0.79" "0.84" "-999" ...
 $ notes        : chr  "ok" "" "scale odd" "not measured" ...

💡 Read the types before you read the numbers

chr where you expected a number is a message from R, not a detail. It means that somewhere in that column is a value R could not read as a number, and str() is showing you the first few so you can see which. Make this the first thing you check on any dataset you are handed, including your own: it takes one line and it catches the problem while it is still cheap.

There is a quieter problem in the same output. trt came back as int, and R is perfectly happy, because 1, 2 and 3 really are integers. But they are not quantities: they are labels for three treatments. A type can be technically correct and scientifically wrong, and R will never warn you about that one. Only the documentation can, which is why Pair 3’s job matters.

6.2.7.5.2.1 Quick Checks on Any data.frame

These are the first things to run on a table somebody hands you, including one you made yourself last year. All of them are base R, so there is nothing to install, and none of them takes more than a second.

Function What it answers
str(d) everything at once: size, column names, types, first few values. Start here
dim(d), nrow(d), ncol(d) how big is it
names(d) what are the columns called
head(d), tail(d) what do the first and the last rows look like. The last rows are where a truncated or half-finished file shows itself
summary(d) per column: range, quartiles and mean if numeric; length and type if not
unique(d$col) every distinct value in one column. This is how you find non-atomic values, and every spelling of missing
table(d$col) how many times each value occurs. Add useNA = "ifany" to count the missing ones too
colSums(is.na(d)) how many missing values per column, as R understands missing
sum(duplicated(d)) how many rows are exact repeats of an earlier row
View(d) opens it as a spreadsheet in RStudio, for when you simply want to look

Two of those deserve running right now, because on this dataset they reveal something the others do not.

First, every distinct value in one column:

# What is actually IN this column?
unique(d$mass)
> unique(d$mass)
 [1] "12.5 g" "11,8"   "12.5"   "-999"   "9.2"    NA       "10.4"   "10.2"
 [9] "1,250"  "10.9"   "8.7"    ""

One line, and the whole column is laid bare. Twelve measurements of body mass, and among them a value carrying its unit, a decimal comma, a thousands separator, a sentinel -999, an empty string and an NA. You have just found most of Pair 2’s work, and you found it without opening the file in anything.

Second, how much of it R thinks is missing:

# How many missing values per column?
colSums(is.na(d))
> colSums(is.na(d))
           V1            x2          temp           trt          mass
            0             0             0             0             1
biomass_index         notes
            0             0

💡 R believes there is exactly ONE missing value in this table

There are at least four, and they are written four different ways. is.na() finds only the one that arrived as NA. To R, -999 is a perfectly good number, an empty string is a perfectly good piece of text, and n/a is a word. None of them is missing as far as R is concerned, and nothing will warn you.

This is the most expensive defect in the whole folder, because it is the one that changes published numbers rather than merely wasting your afternoon. A missing value that R cannot see gets included in the mean. Average that mass column once it has been forced to numeric and -999 is in there, dragging the result down, silently, with no error and no warning. That is how a wrong number reaches a paper.

The fix is not clever code. It is one documented missing-value code, written down where a reader will find it, which is where Pair 3 comes in.

6.2.7.5.2.2 How to Record a Missing Value So That R Sees It

R has exactly one missing value, and it is NA. Everything else is data. This is what read.csv() makes of the codes people actually use:

What is in the file What R makes of it
NA missing. The only code read.csv() recognises by default
an empty field an empty string "" in a text column, which is not missing
-999 a number, and it will go into your mean
n/a, unknown, . a word, which also forces the whole column to text

So there are two ways to get this right.

Recording your own data: write NA. Not blank, not -999, not a word. One code, used for every missing value in the dataset, whatever the reason it is missing.

Inheriting somebody else’s: tell read.csv() what their code was, which you can only do once you have found out, which is what unique() above is for:

# Everything listed here is read as NA instead of as data
d <- read.csv("Messy_Project/copy of data(1).csv", na.strings = c("NA",
    "", "-999", "n/a", "."))

Then say so in the documentation. A dataset with four missing-value codes effectively has none, because a reader cannot tell a measurement from a placeholder, and neither can R. Recording why a value is missing is a separate job: it belongs in a notes column or the data dictionary, not in the measurement cell. -999 in a mass column is somebody trying to store a reason where a number should be.

One last thing worth noticing, since you will reach for it constantly: run summary(d) and look at what it says about mass. It reports Length:12, Class :character, and no mean, no range, no quartiles, because you cannot take the mean of words. The absence of the statistics is the signal. When summary() goes quiet about a column you expected numbers from, that column is text, and you now know why.

6.2.7.5.3 Step 3: Build Your Defect Log (together, 8 minutes)

Whatever your section asks, you are producing one thing: a written record. Not a tidied folder, and not a mental note.

You record it the way this course records everything: data in a file, document generated from it. You built two files before class, and they work together:

  • Defect_log.csv holds the findings, one row per defect, in five columns: file, defect, repairable, fix_pseudocode, practice. Edit it in RStudio, or in a spreadsheet if you save it back as csv.
  • Defect_log.Rmd is the document you knit and hand in. It reads the csv and renders it, so you never type a finding twice and the document cannot disagree with the data.

The chunk that does it is the one you wrote in Insert Tables: read.csv() into a data frame, then kable() to render it. Add a row to the csv, re-knit, and the table updates itself, which is the whole principle of this course in miniature: the report is derived from the data, not transcribed from it.

Spend the first five minutes of this step building the two files. Build them, do not be given them: this is the same work as every R Markdown document you have produced since Chapter 2, and it is quick.

Defect_log.csv, via File > New File > Text File, saved in Chapter_5/, containing one line to start with:

file,defect,repairable,fix_pseudocode,practice

Defect_log.Rmd, via File > New File > R Markdown..., saved beside it, and built so that it:

  1. has a YAML header with a title, both your names, and output: html_document (Chapter 2);
  2. has a setup chunk setting global options and loading knitr, kableExtra and magrittr (Chapter 2);
  3. reads the csv with read.csv() and renders it with kable() and kable_styling(), as in Insert Tables;
  4. counts the defects with inline code;
  5. knits without error.

Knit it now, while the log is still empty. A document that works on an empty table will work on a full one, and a document first knitted at the end of class will fail at the worst possible moment. If it will not knit, ask me now rather than at the report.

✅ Solution: the two finished files

Rather than print them here, take them as files. The .csv is the header row plus one worked example; the .Rmd is complete and knits as it stands.

On each page use the Download raw file button, and save both into your Chapter_5 folder beside Messy_Project/. Knit Defect_log.Rmd once after downloading: it reads the csv, renders it with kable() and counts the rows with inline code, so a clean knit confirms the whole chain works.

6.2.7.5.3.1 Generate the Metadata for This Dataset

A log says what is wrong. Metadata say what the data are, and this project has none: that absence is itself one of the largest defects in the folder. What you build here is the data-level half of it, a row per variable, which is what Data-Level Documentation calls a data dictionary.

Let R build the half it can. Everything in the first five columns comes from the functions you ran in Step 2. R can describe the structure; only a person can supply the meaning. Add this to your script and run it:

# Build a data dictionary skeleton FROM THE DATA ITSELF. R
# can fill the left-hand columns. It cannot fill the
# right-hand ones.
dictionary <- data.frame(variable = names(d), type = sapply(d,
    class), n_missing = colSums(is.na(d)), n_distinct = sapply(d,
    function(x) length(unique(x))), example = sapply(d, function(x) as.character(x[1])),
    meaning = "", units = "", evidence = "", confidence = "",
    row.names = NULL, stringsAsFactors = FALSE)
# Name the dictionary after the file it documents, with the
# awkward characters taken out: inheriting a bad name is
# how bad names spread.
stem <- sub("\\.csv$", "", basename(datafile))
stem <- gsub("[^A-Za-z0-9]+", "_", stem)
stem <- gsub("^_|_$", "", stem)
write.csv(dictionary, paste0("README_", stem, ".csv"), row.names = FALSE)

This is what comes back:

       variable      type n_missing n_distinct example
1            V1   integer         0         12       1
2            x2 character         0          4 3/12/25
3          temp character         0          7    25°C
4           trt   integer         0          3       1
5          mass character         1         12  12.5 g
6 biomass_index character         0         11    0.84
7         notes character         0          9      ok

Read it before you fill it in. trt is an integer with three distinct values: three labels stored as numbers, which nothing will warn you about. x2 has twelve rows and four distinct values, which is what three date formats look like from the outside.

You end up with README_copy_of_data_1.csv, named after the file it documents so nobody has to guess which dataset it belongs to, and cleaned of the spaces and brackets rather than inheriting them. Now fill the four empty columns by hand, for the variables your pair owns:

Column What goes in it
meaning what the variable is, in one line
units units, or the allowed values for a code
evidence where you learned it: a file, a line of a script, a sentence in notes.txt
confidence high, medium, or none

The evidence column is the point. Some you can fill: mass carries g in one value, and notes.txt says -999 = didnt measure. Some you cannot: trt is defined nowhere, because notes.txt says only “trt as agreed w/ SB” and SB is unreachable; biomass_index was “calculated in the spreadsheet” and that formula is gone.

Every cell you cannot fill is a row for your Defect_log. A dictionary you are unable to complete is not a failed exercise: it is the diagnosis, and it is the most precise statement you can make of what this project lost.

Then move on to your own section below. From here on, every defect you find goes into the csv as a row.

6.2.7.5.4 Step 4: Diagnose the Area Your Pair Owns (your pair, 20 minutes)

Now open things. Work only on your own area: if four pairs read the same files you will all find the same three obvious problems and miss everything else.

Twenty minutes, and one output: rows in your csv. Roughly:

5 min Open the files your brief names, and look
12 min Write a row for every defect you can evidence what a good row looks like
3 min Re-knit the .Rmd and read your own table back

Aim for four or five rows you can defend, not fifteen you cannot. A row somebody else could act on is worth more than a list of impressions.

6.2.7.5.4.1 What a Good Row Looks Like

The csv arrives with one row already in it, which is both the format and the standard:

file defect repairable fix_pseudocode practice
copy of data(1).csv mass holds a unit inside the value, 12.5 g, so the whole column imports as text Yes, but every value needs checking see pseudocode below Data Digitization: atomisation, one value per cell

The standard being: be specific enough that somebody who has never seen this folder could act on your row. “File names are bad” is not a row. Five things make a row useful:

Column What it must say The test
file Name the file. Point at it, do not gesture at the folder could a stranger open it?
defect What is wrong, concretely. Quote the offending value or name is the evidence in the cell?
repairable Yes, or gone for good. Saying precisely which is the more valuable answer, because it is the one that changes how people work would you bet on it?
fix_pseudocode How you would fix it, in pseudocode. The most useful column in the log could somebody else implement it?
practice The practice that would have prevented it, with the section linked. This is where your charter clause comes from in the reporting step does it name a practice, not a preference?
6.2.7.5.4.2 Write the Fix as Pseudocode (how to fill the fix_pseudocode column)

Specifying a repair does not require knowing which R functions would carry it out, which is just as well: the R for most of these is in Chapter 7, which is reference material you work through for your own project rather than a class we sit through. Pseudocode, which you met in Chapter 2, is a plain-language statement of the logic: the steps, in order, in the structure a program would take, written for a human rather than a machine. It is the right output for today because it separates deciding what must happen from knowing which function does it, and the first of those is the hard part.

So for any defect where you can, add a numbered procedure. For the mass column, it might run:

FOR the mass column:
  1. strip any unit out of the value            "12.5 g"  -> "12.5"
  2. replace a decimal comma with a point       "11,8"    -> "11.8"
  3. remove any thousands separator             "1,250"   -> "1250"
       ?? CHECK: is 1,250 a mass of 1250, or of 1.250 written badly?
          It is 100x heavier than every other animal. ASK. Do not guess.
  4. convert every agreed missing code to NA    -999, "", "n/a", "unknown"
  5. convert the whole column to numeric
  6. CHECK: count the NAs. It must equal the number you counted by hand
       in Step 2. If it does not, step 1-4 silently ate a real value.

💡 Notice what writing that forced you to do

Three things, and none of them is about R.

It made you put the steps in an order. Converting to numeric before stripping the units destroys the data; the sequence is part of the fix, not a detail.

It exposed a decision the data cannot settle. Step 3 is not a programming problem. No function will tell you whether 1,250 is a thousand-fold error or a misplaced decimal, and a pseudocode step that says ASK is a better answer than any line of code, because the alternative is a silent guess inside a published analysis.

It made you specify how you will know it worked. Step 6 is a test. Writing the check at the same time as the procedure is a habit worth forming now: it is the same instinct as the defensive programming you wrote in Chapter 2 - PART E.

Where you cannot write pseudocode, say so and say why. “No procedure exists: the treatment key was never written down, so no amount of code recovers what 1, 2 and 3 meant” is a complete and valuable entry. It is also the entry that will matter most in Step 4.

6.2.7.5.4.3 Pair 1: Naming and Organisation
  • There is more than one file that could be “the data”. List them. Using the file names alone, can you establish which is authoritative? If not, say what naming convention would have made it obvious.
  • Write down everything wrong with the name Data final v2 FINAL.xlsx. There are at least four distinct problems.
  • Two files are dated in their names. What date does plot data 12-03-25.csv refer to? Defend your answer.
  • There are three scripts. Judging by their names only, which would you run? Now open all three and say whether your choice was right.
  • The folder is flat. Propose the directory structure this project should have had, and say where each existing file goes in it. Compare your answer with the worked example in Data Structure and Organisation of Files.
  • Write down the naming convention you would impose, in one sentence, and show what three of these files would be called under it. Do not actually rename anything: the folder has to stay in the state it arrived in.
6.2.7.5.4.4 Pair 2: Formats and Atomisation

Start with the format itself. The master file is .xlsx. Give two reasons from Processing Data why that is the wrong format for archiving, then open it and find a third problem that would break read.csv() even after conversion.

Then look inside. Load the csv and try to use the mass column as a number:

d <- read.csv("Messy_Project/copy of data(1).csv")
d$mass
as.numeric(d$mass)

How many of the twelve values survive, and why does each loss happen? Account for every one. Then explain why as.numeric() giving you a result at all is more dangerous than it refusing.

Then work through the rest of the file:

  • Find the cell that contains two data points. Find the cells that contain a unit. Find the cells that use a comma as a decimal separator.
  • How many different ways is “missing” encoded in this dataset? List them all, and say which R will understand without being told.
  • The x2 column holds dates. How many formats, and is any one of them unambiguous?
  • trt holds 1, 2 and 3. What would go wrong if you fitted lm(mass ~ trt) as the data stand? (Open analysis_USE THIS ONE.R: somebody already did.)
  • List what would have to change for this table to be analysable: one atomic value per cell, one missing-value code, one date format, units moved into the column names, and a format that will still open in twenty years. You are listing, not repairing. Doing it in R is the exercise we build in Chapter 7.
6.2.7.5.4.5 Pair 3: Documentation and Metadata
  • What are the three treatments? The trt column of copy of data(1).csv holds 1, 2 and 3. What do those three numbers mean? Find the answer, or establish that it cannot be found.
  • Fill in the meaning, units, evidence and confidence columns of README_copy_of_data_1.csv for every variable, not just yours. Which ones can you not fill, and what would you have needed?
  • biomass_index is a derived variable. From which columns, by what formula? Can you reproduce a single one of its values?
  • The missing-value code is recorded somewhere in this folder. Find it, and then explain why where it is recorded makes it almost useless.
  • notes.txt contains at least three facts that change how the data must be analysed. Identify them, and say where each one should have been recorded instead.
  • Using Project-Level Documentation and Data-Level Documentation as your outline, sketch the README.md this project needs: the headings, and whatever you can genuinely fill in. Mark everything you cannot with UNRECOVERABLE. The list of UNRECOVERABLE items is your real finding, and unlike most of what the other pairs report, no amount of R will ever bring those back.
6.2.7.5.4.6 Pair 4: Preservation and Sharing
  • Which of the five types of research data is this? Your answer determines how much everything else in this list matters, so settle it first.
  • Look at the last line of analysis_USE THIS ONE.R. What does that script do to its own input, and what has been permanently lost as a result? Relate your answer to the rule raw data should remain raw.
  • How many copies of these data exist, and in how many places? Name the single event that ends this project.
  • Open sites_landowners.csv. May this file be shared? Work through When Data Can’t Be Shared and say which grounds apply, to which columns. Then connect it to the CARE Principles and Data Sovereignty from Chapter 4.
  • Sort every file in the folder into three piles: deposit openly, deposit with restricted access, never leaves the lab. Justify the boundaries.
  • Choose a repository for the depositable material and state what it would require of you that this project cannot currently supply.

6.2.7.6 Reporting

You do not report today. Reporting is the end of this activity, and it happens in the PART B class, as Steps 3 to 5 of Read the Code You Inherited: one round, in which your data defects and that project’s code defects are reported together. The gap is deliberate: it gives you a few days with the log, and a diagnosis you can still defend after sleeping on it is worth more than one delivered while the folder is still open in front of you.

Bring two things to that class: the knitted Defect_log and your group’s charter.

6.2.7.6.1 What your pair will report

About two minutes, to the whole class:

  1. Which area you owned, and how many defects you logged in it.
  2. The defect that mattered most: name the file, quote the evidence, and say why this one rather than another.
  3. What is repairable and what is permanently lost. The second half is the more valuable answer, because it is the one that changes how people work in future.
  4. The practice that would have prevented it, which is the column of your log that becomes a charter clause in the next step.
6.2.7.6.2 The three steps that follow, all in the PART B class

They are Steps 3 to 5 of that class’s own activity, and they cover both diagnoses at once: what you found in these data, and what the same project’s code turns out to have done with them.

Step What you do Time
5 Report What You Found The four points above, and then your code defect 20 min
6 Write the Clause One clause for the data defect, one for the code defect 15 min
7 Merge Into Your Charter The two pairs in each group reconcile all eight clauses 10 min, at home
6.2.7.6.3 What you leave with, and where it goes

Look down your knitted Defect_log before you leave. Most of the rows describe something you can now see clearly and cannot yet do anything about: a column of measurements that imports as text, four different ways of writing “missing”, three date formats, a derived variable with no formula.

That is the right place to be, and it is not an accident. Noticing is the hard part and the part that is usually missing; the procedures are learnable once you know what you are looking for. Having spent a class watching as.numeric() quietly destroy five of twelve measurements, you now have a concrete reason to want the technique that prevents it, which is a far better position than meeting the technique first and wondering when it would ever matter.

So keep the log, and keep the folder exactly as it is while we are using it. Chapter 7 is where the procedures are written up, and it is yours to work through: importing data that does not want to be imported, finding and fixing the values R has misread, settling on one missing-value code, and getting a table into a shape an analysis can trust.

Your log is the task list for those exercises, and your pseudocode is the specification. That is the ordinary order of work: decide what has to happen, write it down in a form a human can check, and only then find the functions that do it. Turning your own pseudocode into working R is a much better exercise than following somebody else’s, because you already know why every step is there.

What this folder was hiding (open this after class, not before)

Every defect was planted, and this is the full inventory. If your pair found its own four, you did well; nobody is expected to find all of these in fifteen minutes.

Naming and organisation. Three rival versions of one dataset, with no master copy identifiable; Data final v2 FINAL.xlsx (spaces, two conflicting version markers, no sequence); copy of data(1).csv (parentheses, “copy of”, silently differs from the xlsx in two rows); plot data 12-03-25.csv (unresolvable: 12 March or 3 December); three analysis scripts of which the authoritative-sounding one is the worst; a flat folder with no separation of data, code and outputs.

Formats and atomisation. Proprietary .xlsx master; a title row above the column names, so the header is not on row 1; 12.5 g, male in a single cell; units inside values (12.5 g, 25°C); decimal commas (11,8) and a thousands separator (1,250); non-ASCII characters including ±, ° and curly quotes; four missing-value codes (-999, empty, NA, n/a, plus .); three date formats; a categorical treatment coded as integers and then fitted as a continuous predictor.

Documentation and metadata. No README, no data dictionary, no licence, no protocol, no contact information; column names V1, x2, trt; no units anywhere; the treatment key written down nowhere, which is unrecoverable; the missing-value code recorded only in a private note; biomass_index derived “in the spreadsheet”, with no formula and no way to reproduce it; three analysis-critical facts (an uncalibrated balance, unrecorded juveniles, a resampled site) surviving only as shorthand in notes.txt.

Preservation and sharing. Observational data, therefore irreplaceable; one copy in one folder; a script whose final line overwrites its own input, so the raw file is already gone; personal names, phone numbers and the exact coordinates of a federally Threatened species sitting unprotected beside everything else; no licence, no repository, no DOI, no data availability statement; an absolute setwd() that guarantees the scripts run on exactly one computer, which no longer exists.

6.3 Reproducible Code

6.3.1 Aim

This section of Chapter 5 aims to introduce students to the principles and practices of reproducible coding. It provides guidance on organizing projects, maintaining clean and transparent code, and managing software dependencies to support reproducible research workflows.

This part runs over two classes, and most of it is group work. In PART A you judged the data of a project you inherited. Here you judge its code, and it is the same project: three R scripts that nobody in the room has yet read a line of.

💡 Two things to do before class

  1. Read the four sections on reproducible code: project organisation, programming standards, code portability and software dependencies. My time in class goes on applying them, not on introducing them. The list, with links, is in the Before Class Activity.
  2. Start a second script and diagram the project you already have. This is a short piece of hands-on work, not reading. Keep Messy_Project exactly as it is, and bring the Defect_log.Rmd and Defect_log.csv you built in PART A: you add to them in class.

Before class

Block Time Who
The four sections, read in advance 25 min Students, at home before class
Set up your script and diagram the project 10 min Students, at home before class

Class 1 — the workflow, and reading the code you inherited

Block Time Who
One Pass Over the Workflow, with the slides 12 min Instructor
👥 Step 1: Try to Run It 7 min Whole class
five-minute break 5 min
👥 Step 2: Diagnose the Area Your Pair Owns 15 min Students, in 4 pairs
Total in class 39 min

Step 2 is finished at home. Reading three scripts against a standard and writing down what fails takes longer than a quarter of an hour, and the rest of the session goes on the questions it raises.

Class 2 — reporting, and the clauses

Block Time Who
👥 Step 3: Report What You Found, data and code defects together 20 min Students, in 4 pairs
👥 Step 4: Write the Clauses, one for the data and one for the code 15 min Students, in 4 pairs
Total in class 35 min

After class

Block Time Who
👥 Step 5: Merge Into Your Charter 10 min Students, in 2 groups, at home

Step 5 is done at home, because your charter is a live document and the clauses are copied out of the Clauses sheet rather than written from scratch. We go over the charters briefly at the start of the next class, which then begins Chapter 6 with the three licence families.

6.3.2 Introduction

To produce reproducible code, the following steps must be integrated:

  • Step 1: Establish a reproducible project workflow.
  • Step 2: Organize the project for reproducibility.
  • Step 3: Ensure basic programming standards.
  • Step 4: Document and manage software dependencies.
  • Step 5: Produce a reproducible report (using R Markdown).
  • Step 6: Implement version control (using Git).
  • Step 7: Archive and cite code appropriately.

In this chapter, we will cover Steps 1 to 4. Step 5 was introduced in Chapter 2, and Step 7 is discussed in Chapter 5 under Data Management. Step 6 will be addressed in the bioinformatics tutorial associated with Chapter 12.

6.3.3 Learning Outcomes

By the end of this part, students are able to:

💡 These say judge and diagnose on purpose

As in PART A, this class is about detecting problems rather than fixing them. You will specify every repair in pseudocode, and implement it in your own project, with Chapter 7 as the reference. Reading code critically is the skill that makes the fixing worth anything, and it is the one nobody is taught.

6.3.4 Before Class Activity

6.3.4.1 What to Read Before Class (25 minutes)

Read these four sections before class. They all sit under Resources below, and the class assumes you have read them: the time in the room goes on applying them, not on introducing them.

  1. Organize Your Project for Reproducibility — what a script-based workflow looks like, the principles behind a good one, and how a project’s folders should be laid out
  2. Apply Basic Programming Standards — naming, pseudocode, style, comments, functions and testing. This is the longest and the most important of the four
  3. Code Portability — absolute against relative paths, and why one of them stops your code running on anybody else’s machine
  4. Document and Manage Software Dependencies — what sessionInfo() is for

💡 One short extra

Software Licenses is half a page and worth reading at the same time, because a project nobody is allowed to reuse is not reproducible in any sense that matters.

6.3.4.2 What to Set Up on Your Computer Before Class (10 minutes)

You already have the project: Messy_Project, in your Chapter_5 folder from PART A. Do not download it again, and do not tidy it. Two things to do.

6.3.4.2.1 Step 1: Start a Second Script

In RStudio, File > New File > R Script, and save it in Chapter_5/ as:

`Chapter_5_reproducible_code.R`

It sits beside Chapter_5_data_management.R from PART A. Two scripts, one folder, one copy of the project: each script does a different job and says so in its name, which is the naming practice from Informative, Consistent Naming applied to your own work.

Then point R at the folder, exactly as before, and check rather than assume:

# Session > Set Working Directory > To Source File
# Location, then:
getwd()  #must end in Chapter_5
6.3.4.2.2 Step 2: Diagram What You Inherited

Work through Now Diagram the Project You Inherited in your new script. It is a handful of lines, it needs the data.tree and DiagrammeR packages from R Packages, and it produces the picture we open class with.

✅ Checkpoint: Chapter_5/ holds two scripts and one Messy_Project/, getwd() ends in Chapter_5, and you have seen your own version of Figure 6.7.

6.3.5 Class Outline

PART B runs over two classes, and the activity is split between them.

Class 1

  1. One Pass Over the Workflow (12 minutes, instructor, with slides)
  2. 👥 Group Activity: Read the Code You Inherited, Step 1 and Step 2 (22 minutes; Step 2 is finished at home)

Class 2

  1. 👥 The same activity, Steps 3 to 5 (45 minutes; this round also closes PART A, and it is the end of Chapter 5)
  2. Then Chapter 6 begins

Everything else in this part is reference material, gathered under Resources below. Your pre-reading list says which of it to read before class.

6.3.6 Resources

Everything you need to read for this part is in this section, as the subsections below. Your pre-reading list says which of them to read before class.

This part draws on the following resources:

6.3.6.1 Teaching Material

The presentation associated with this class is available here:

💡 Best browser to view presentation

  • For optimal viewing of this presentation in full-screen presentation mode, the instructor recommends using Mozilla Firefox. Its built-in PDF viewer supports seamless full-screen display with easy navigation tools, making it ideal for presentations without the need for external plugins. Other browsers may not handle PDF presentation features as reliably.

6.3.6.2 Data and Dependencies

To complete the exercise on using R to see a project’s structure you need two things:

  1. The project folder, which you already have from PART A. Nothing to download.
  2. Two R packages, which you may need to install.

Details for each are below.

6.3.6.2.1 Data

You work on the same folder you inherited in PART A: Messy_Project, sitting in Chapter_5/. There is nothing new to download, and that is deliberate. You have already judged this project’s data; this part judges its code, and the two are the same project because in real life they always are.

If you do not have the folder, the two lines of R that fetch it are in What to Set Up on Your Computer Before Class for PART A.

💡 And one folder to look at, but not to download

This part also refers to Project_ID, a small project laid out the way one should be (Figure 6.4). It lives in the course repository, and the diagrams below are generated from it when this page is built, so you can read them here without downloading anything. Use it as the comparison: Project_ID is the target, Messy_Project is what you have.

6.3.6.2.2 R Packages

To ensure you have all the necessary R packages, we will reuse code introduced in Chapter 1. The exercises depend on the following packages:

  • data.tree
  • DiagrammeR

Please make sure these packages are installed and loaded in your R environment before proceeding. You can use the R code provided below to do this.

Before running the code, please read the disclaimer:

💡 Disclaimer

  • Warning: The code in this section will install the following R packages (and their dependencies) on your computer: data.tree and DiagrammeR.
  • About the R packages: These packages help you organize your project for reproducibility. Both are available on CRAN, the official repository for R packages.
  • R package repositories: Before running the installation code, ensure your R package repositories are properly set in RStudio. You can follow this guide for detailed instructions.
  • Questions or concerns: If you need assistance, please contact the instructor at svenbuerki@boisestate.edu.

The code to install R packages is as follows:

## ~~~ 1. List all required packages ~~~ Object (args)
## provided by user with names of packages stored into a
## vector
pkg <- c("data.tree", "DiagrammeR")
## ~~~ 2. Check if pkg are installed ~~~
print("Check if packages are installed")
## [1] "Check if packages are installed"
# This line outputs a list of packages that are not
# installed
new.pkg <- pkg[!(pkg %in% installed.packages())]
## ~~~ 3. Install missing packages ~~~
if (length(new.pkg) > 0) {
    print(paste("Install missing package(s):", new.pkg, sep = " "))
    # repos= names the CRAN mirror explicitly, so the
    # install never stops to ask which mirror to use. This
    # matters when knitting: a prompt has nowhere to appear
    # and the document fails. cloud.r-project.org is CRAN's
    # global mirror.
    install.packages(new.pkg, dependencies = TRUE, repos = "https://cloud.r-project.org/")
} else {
    print("All packages are already installed!")
}
## [1] "All packages are already installed!"
## ~~~ 4. Load all packages ~~~
print("Load packages and return status")
## [1] "Load packages and return status"
# Here we use the sapply() function to require all the
# packages
sapply(pkg, require, character.only = TRUE)
##  data.tree DiagrammeR 
##       TRUE       TRUE

6.3.6.3 Organize Your Project for Reproducibility

6.3.6.3.1 Visualize Your Reproducible Project Workflow

A key step to organizing your project for reproducibility is to create a clear workflow diagram that outlines how your project will be structured and executed. An example of such a reproducible project workflow is shown in Figure 6.3. We will use this workflow as a guiding template throughout this chapter to help you implement your own reproducible project.

Four grey text boxes in a clockwise loop joined by arrows: Before starting (folder structure, naming, coding style, install Git), then First steps (create folders, README, version control repo, LICENSE, reproducible report), then Write reproducible code, an inner top-to-bottom chain from write pseudocode to write code, program defensively, comment, test, and document code, then Prepare for publication (record package versions, update README, get colleague review, make repo public, archive with DOIs).

Figure 6.3: A simple reproducible project workflow.

6.3.6.3.2 One Pass Over the Workflow (12 minutes)

This is the part of the class I lecture, using the slides in Teaching Material, and it is the spine of everything that follows. You have read the four sections before class, so this is a synthesis rather than an introduction. Each row below is one stage of Figure 6.3. For each I give you what the stage is for, the failure it invites, and where you are about to meet that failure in the project you inherited.

The seven steps listed in the Introduction map onto these stages; this part of the course covers Steps 1 to 4.

Stage What it is for The failure it invites Where you will meet it
Before starting Settling the conventions once, before a single file exists: folder layout, naming, coding style Every file gets named and placed ad hoc, and nobody can tell which of several files is authoritative three rival scripts, one of them called analysis_USE THIS ONE.R
First steps Building the skeleton: folders, a README, a licence, version control, a report that regenerates A flat folder nobody can navigate, with no statement of what may be reused the one-level diagram in Figure 6.7, and no LICENSE anywhere
Write reproducible code Code a human can follow and a machine can re-run: pseudocode first, relative paths, defensive checks, comments on the why, tests Code that runs on exactly one machine, destroys values in silence, and cannot be read six months later setwd() to a dead laptop on line 1, and as.numeric() with nothing checking what it ate
Prepare for publication Making the result checkable by somebody else: versions recorded, README current, archived with a DOI An analysis nobody can rebuild, even holding every file no sessionInfo(), no record of a single package version

💡 Why the first stage is called before starting

Look at the fourth column. Every defect you are about to find was cheaper to prevent than to discuss, and most of them belong to the first two stages, before any analysis was attempted. Deciding a naming convention costs one conversation. Deciding it after three scripts exist means somebody has to work out which one is real, and in this project nobody can.

This is the same argument as Plan sitting at the hub of the data life cycle in PART A, arriving from the other direction. Reproducibility is overwhelmingly a set of decisions taken early and cheaply, rather than a repair effort at the end.

6.3.6.3.3 The Repeatable, Reproducible Analysis Workflow

The fundamental idea behind a robust, reproducible analysis is a clean, repeatable, script-based workflow (i.e. the sequence of tasks from the start to the end of a project), linking raw data through to clean data and final analysis outputs.

6.3.6.3.4 Principles of a Good Analysis Workflow

Please find below some key concepts associated with this task:

  • Start your analysis using copies of your raw data.
  • Any cleaning, merging, transforming, etc. of data should be done using scripts, not manually.
  • Split your workflow (scripts) into logical thematic units. For example, you might separate your code into scripts that
    1. load, merge, and clean data,
    2. analyze data,
    3. produce outputs such as figures and tables.
  • Eliminate code duplication by packaging useful code into custom functions (see Step 3: Ensure basic programming standards). Make sure to comment your functions thoroughly, explaining their expected inputs and outputs (as well as the associated arguments and their options), and what they do and why.
  • Document your code and data using comments within scripts and by producing separate documentation (using the R Markdown format).
  • Any intermediate outputs generated by your workflow should be kept separate from raw data.
  • Keep your raw data raw.
6.3.6.3.5 Organizing and Documenting Workflows

The simplest and most effective way to document your workflow—its inputs and outputs—is through good file system organization and informative, consistent naming of materials associated with your analysis. The name and location of files should be as informative as possible regarding what a file contains, why it exists, and how it relates to other files in the project. These principles apply to all files in your project (not just scripts) and are also closely linked to good research data management (see Chapter 5: Data Management).

6.3.6.3.6 File System Structure

It is best to keep all files associated with a particular project in a single root directory. RStudio Projects offer a great way to keep everything together in a self-contained and portable manner (i.e. so files can be moved from computer to computer), allowing internal pathways to data and scripts to remain valid even when shared or relocated.

There is no single best way to organize a file system. The key is to ensure that the structure of directories and the location of files are consistent, informative, and tailored to your workflow.

The course repository carries a worked example, and you should go and look at it. Project_ID is a small project laid out the way one should be. You can browse it on GitHub, and its own README.md explains what each directory holds and why it is where it is. Its structure is drawn in Figure 6.4, generated from the real folder when this page is built, so the figure cannot drift out of step with the example it describes. The convention it follows is set out in Numbering the Flow, immediately below.

6.3.6.3.7 Numbering the Flow

Figure 6.4 already numbers the scripts: 01_download_data.R through 05_generate_figures.R, so that reading them in order is reading the analysis in order. Number the directories on the same principle and the folder listing becomes the pipeline.

Project_ID/
├── README.md                 <- what this project is, who to ask, how to run it
├── LICENSE                   <- what others may do with it
├── 01_Raw_Data/              <- exactly as collected. WRITE ONCE, NEVER EDIT
├── 02_Processed_Data/        <- written only by scripts in 03_Scripts/
├── 03_Scripts/
│   ├── 01_import_data.R
│   ├── 02_clean_data.R
│   ├── 03_exploratory_analyses.R
│   ├── 04_fit_models.R
│   └── 05_generate_figures.R
├── 04_Output/                <- fitted models, intermediate results
├── 05_Figures_and_Tables/    <- what goes into the manuscript
├── 06_Reports/               <- R Markdown documentation of the analysis
├── 07_MS/                    <- the manuscript itself
└── R_functions/              <- NOT numbered: see below

The number means position in the flow, not importance. Reading that listing from top to bottom is reading the project from the field to the journal, and three useful things follow:

  • A newcomer finds the entry point without asking anybody. 01_ is where the data came in; the analysis goes forward from there.
  • “Where does this file go?” has exactly one answer, which is the whole purpose of a convention. Compare that with the project you inherited, where three scripts and three near-identical data files sit in one undifferentiated heap.
  • The direction of travel is visible, so a script that writes backwards is obvious at a glance. A script in 03_Scripts/ writing into 01_Raw_Data/ is wrong, and you can see it is wrong without reading a line of it.

Three rules that make the numbering work:

  1. Left-pad single digits with a zero. 01, not 1. Without the zero your listing sorts 1, 10, 2, 3, which is worse than no numbering at all. This is the same warning as the caption of Figure 6.4.
  2. Number the flow and nothing else. README.md and LICENSE are not steps in an analysis, so they do not get numbers. R_functions/ is not numbered either, because it is used at every stage: its contents are called by 02_, 03_ and 04_ alike, so any number you gave it would be a lie.
  3. 01_Raw_Data/ is written once and never again. Not by you, and above all not by a script. If a cleaning step needs to write something, it writes into 02_Processed_Data/. This is the keep your raw data raw rule given a place to live, so that following it is the path of least resistance.

💡 Avoid & and spaces in directory names

The structure above says 05_Figures_and_Tables rather than Figures_&_Tables. An ampersand is a control character in most shells and has to be escaped on the command line; a space has to be quoted. Neither will stop R, and both will eventually stop something else, usually at an inconvenient moment. The convention costs nothing to adopt and nothing to maintain.

💡 A numbered structure is a claim you have to keep

Numbering is a promise that the flow really does run in that order. If 03_ secretly depends on something produced by 05_, the numbers are now actively misleading, and a reader who trusts them will be wrong in a way they cannot detect. Renumber when the workflow changes, and keep the README in step: a convention nobody maintains is worse than none, because it is believed.

6.3.6.4 Using R to Infer Data Structure and File Organization

Inferring the directory tree structure of your project provides a simple and efficient way to summarize the data structure and organization of files related to your project, as well as to track versioning. The R base functions list.files() and file.info() can be combined to obtain information about the files stored in your project.

We do this twice. First on Project_ID, the project laid out the way one should be (Figure 6.4), so that you can see what a healthy structure looks like as a diagram. Then on Messy_Project, the one you inherited, so that you can see what the absence of a structure looks like. The comparison is the exercise.

# Produce a list of all files in working directory
# (Project-ID) together with info related to those files
file.info(list.files(path = "Project_ID", recursive = TRUE, full.name = TRUE))
##                                                              size isdir mode
## Project_ID/01_Raw_Data/species_data.csv                        48 FALSE  700
## Project_ID/02_Processed_Data/species_data_clean.csv            71 FALSE  644
## Project_ID/03_Scripts/01_download_data.R                       14 FALSE  700
## Project_ID/03_Scripts/02_clean_data.R                          14 FALSE  700
## Project_ID/03_Scripts/03_exploratory_analyses.R                14 FALSE  700
## Project_ID/03_Scripts/04_fit_models.R                          14 FALSE  700
## Project_ID/03_Scripts/05_generate_figures.R                    14 FALSE  700
## Project_ID/04_Output/DataStr_network.html                  256368 FALSE  700
## Project_ID/04_Output/DataStr_tree.pdf                        3611 FALSE  700
## Project_ID/05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf  10282 FALSE  700
## Project_ID/06_Reports/Documentation.md                         14 FALSE  700
## Project_ID/07_MS/MS_et_al.docx                              11705 FALSE  700
## Project_ID/LICENSE                                            260 FALSE  644
## Project_ID/R_functions/check.install.pkg.R                    960 FALSE  700
## Project_ID/README.md                                         1505 FALSE  700
##                                                                          mtime
## Project_ID/01_Raw_Data/species_data.csv                    2026-10-09 15:44:36
## Project_ID/02_Processed_Data/species_data_clean.csv        2026-10-09 15:44:36
## Project_ID/03_Scripts/01_download_data.R                   2018-09-12 09:56:42
## Project_ID/03_Scripts/02_clean_data.R                      2018-09-12 09:56:42
## Project_ID/03_Scripts/03_exploratory_analyses.R            2018-09-12 09:56:42
## Project_ID/03_Scripts/04_fit_models.R                      2018-09-12 09:56:42
## Project_ID/03_Scripts/05_generate_figures.R                2018-09-12 09:56:42
## Project_ID/04_Output/DataStr_network.html                  2019-09-23 14:27:30
## Project_ID/04_Output/DataStr_tree.pdf                      2019-09-23 14:04:03
## Project_ID/05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf 2018-09-09 11:38:23
## Project_ID/06_Reports/Documentation.md                     2018-09-12 09:56:42
## Project_ID/07_MS/MS_et_al.docx                             2019-09-23 12:51:58
## Project_ID/LICENSE                                         2026-10-09 15:44:36
## Project_ID/R_functions/check.install.pkg.R                 2026-09-18 09:05:58
## Project_ID/README.md                                       2026-10-09 15:44:36
##                                                                          ctime
## Project_ID/01_Raw_Data/species_data.csv                    2026-10-09 15:44:36
## Project_ID/02_Processed_Data/species_data_clean.csv        2026-10-09 15:44:36
## Project_ID/03_Scripts/01_download_data.R                   2026-10-09 15:43:59
## Project_ID/03_Scripts/02_clean_data.R                      2026-10-09 15:43:59
## Project_ID/03_Scripts/03_exploratory_analyses.R            2026-10-09 15:43:59
## Project_ID/03_Scripts/04_fit_models.R                      2026-10-09 15:43:59
## Project_ID/03_Scripts/05_generate_figures.R                2026-10-09 15:43:59
## Project_ID/04_Output/DataStr_network.html                  2026-10-09 15:43:59
## Project_ID/04_Output/DataStr_tree.pdf                      2026-10-09 15:43:59
## Project_ID/05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf 2026-10-09 15:43:59
## Project_ID/06_Reports/Documentation.md                     2026-10-09 15:43:59
## Project_ID/07_MS/MS_et_al.docx                             2026-10-09 15:43:59
## Project_ID/LICENSE                                         2026-10-09 15:44:36
## Project_ID/R_functions/check.install.pkg.R                 2026-09-18 09:05:58
## Project_ID/README.md                                       2026-10-09 15:44:36
##                                                                          atime
## Project_ID/01_Raw_Data/species_data.csv                    2026-10-09 16:19:03
## Project_ID/02_Processed_Data/species_data_clean.csv        2026-10-09 16:19:03
## Project_ID/03_Scripts/01_download_data.R                   2026-10-09 16:19:03
## Project_ID/03_Scripts/02_clean_data.R                      2026-10-09 16:19:03
## Project_ID/03_Scripts/03_exploratory_analyses.R            2026-10-09 16:19:03
## Project_ID/03_Scripts/04_fit_models.R                      2026-10-09 16:19:03
## Project_ID/03_Scripts/05_generate_figures.R                2026-10-09 16:19:03
## Project_ID/04_Output/DataStr_network.html                  2026-10-09 16:19:03
## Project_ID/04_Output/DataStr_tree.pdf                      2026-10-09 15:44:01
## Project_ID/05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf 2026-10-09 15:44:01
## Project_ID/06_Reports/Documentation.md                     2026-10-09 16:19:03
## Project_ID/07_MS/MS_et_al.docx                             2026-10-09 15:43:59
## Project_ID/LICENSE                                         2026-10-10 09:05:09
## Project_ID/R_functions/check.install.pkg.R                 2026-09-18 09:14:54
## Project_ID/README.md                                       2026-10-09 16:19:03
##                                                            uid gid uname grname
## Project_ID/01_Raw_Data/species_data.csv                    502  20  sven  staff
## Project_ID/02_Processed_Data/species_data_clean.csv        502  20  sven  staff
## Project_ID/03_Scripts/01_download_data.R                   502  20  sven  staff
## Project_ID/03_Scripts/02_clean_data.R                      502  20  sven  staff
## Project_ID/03_Scripts/03_exploratory_analyses.R            502  20  sven  staff
## Project_ID/03_Scripts/04_fit_models.R                      502  20  sven  staff
## Project_ID/03_Scripts/05_generate_figures.R                502  20  sven  staff
## Project_ID/04_Output/DataStr_network.html                  502  20  sven  staff
## Project_ID/04_Output/DataStr_tree.pdf                      502  20  sven  staff
## Project_ID/05_Figures_and_Tables/Fig_01_Data_lifecycle.pdf 502  20  sven  staff
## Project_ID/06_Reports/Documentation.md                     502  20  sven  staff
## Project_ID/07_MS/MS_et_al.docx                             502  20  sven  staff
## Project_ID/LICENSE                                         502  20  sven  staff
## Project_ID/R_functions/check.install.pkg.R                 502  20  sven  staff
## Project_ID/README.md                                       502  20  sven  staff
6.3.6.4.1 Pseudocode and Associated R Script

The code presented above could be at the core of a user-defined function designed to manage files and ensure data reliability for your project. To help you define such a function, let’s investigate this further and produce a diagram summarizing your project structure. This objective is achieved in four steps (see Figure 6.4):

  • Step 1: Produce a list of all files in your working directory using the file.info() and list.files() functions. The output of this code is a data.frame.
  • Step 2: Convert the data.frame into a data.tree class object using the data.tree::as.Node() function from the data.tree package.
  • Step 3: Prepare and plot a diagram of the project structure. This is done using a set of functions from data.tree, and the output is plotted using the DiagrammeR package.
  • Step 4: Save the output from Step 3.

💡 Disclaimer

  • Before starting: you need Messy_Project, the folder you downloaded in PART A. Nothing else.
  • Questions or concerns: If you need assistance, please contact the instructor at svenbuerki@boisestate.edu.
6.3.6.4.2 R Code and Associated Outputs

Procedure to follow:

  1. Work in Chapter_5_reproducible_code.R, the script you created before class.
  2. Ensure that the required R packages are installed. See here for more details.
  3. Study the R code below, which infers the file structure of Project_ID/. You do not need the folder on your computer to follow it: the output shown is generated from the copy in the course repository. Read the accompanying text, which explains the approach, then apply the same procedure to your own inherited project in Now Diagram the Project You Inherited.
### Load R packages
library(data.tree)
library(DiagrammeR)
### Step 1: Produce a list of all files in Project_ID
filesInfo <- file.info(list.files(path = "Project_ID", recursive = TRUE,
    full.name = TRUE))
### Step 2: Convert filesInfo into data.tree class
myproj <- data.tree::as.Node(data.frame(pathString = rownames(filesInfo)))
# Inspect output
print(myproj)
##                            levelName
## 1  Project_ID                       
## 2   ¦--01_Raw_Data                  
## 3   ¦   °--species_data.csv         
## 4   ¦--02_Processed_Data            
## 5   ¦   °--species_data_clean.csv   
## 6   ¦--03_Scripts                   
## 7   ¦   ¦--01_download_data.R       
## 8   ¦   ¦--02_clean_data.R          
## 9   ¦   ¦--03_exploratory_analyses.R
## 10  ¦   ¦--04_fit_models.R          
## 11  ¦   °--05_generate_figures.R    
## 12  ¦--04_Output                    
## 13  ¦   ¦--DataStr_network.html     
## 14  ¦   °--DataStr_tree.pdf         
## 15  ¦--05_Figures_and_Tables        
## 16  ¦   °--Fig_01_Data_lifecycle.pdf
## 17  ¦--06_Reports                   
## 18  ¦   °--Documentation.md         
## 19  ¦--07_MS                        
## 20  ¦   °--MS_et_al.docx            
## 21  ¦--LICENSE                      
## 22  ¦--R_functions                  
## 23  ¦   °--check.install.pkg.R      
## 24  °--README.md
### Step 3: Prepare and plot diagram of project structure
### (it requires DiagrammeR)
# Set general parameters related to graph
data.tree::SetGraphStyle(myproj$root, rankdir = "LR")
# Set parameters for edges
data.tree::SetEdgeStyle(myproj$root, arrowhead = "vee", color = "grey",
    penwidth = "2px")
# Set parameters for nodes
data.tree::SetNodeStyle(myproj, style = "rounded", shape = "box")
# Apply specific criteria only to children nodes of Scripts
# and R_functions folders [[ ]] rather than $ because the
# folder name starts with a digit
data.tree::SetNodeStyle(myproj[["03_Scripts"]], style = "box",
    penwidth = "2px")
data.tree::SetNodeStyle(myproj$R_functions, style = "box", penwidth = "2px")
# Plot diagram
plot(myproj)

Figure 6.4: Diagram of project structure for the Project_ID directory. Nodes representing folders and files associated to R code are symbolized by boxes, whereas the others are rounded.

Finally, the R DiagrammeR package unfortunately doesn’t allow you to easily save the graph to a file (Step 4) using, for example, the pdf() and dev.off() functions. However, this task can be accomplished in RStudio as follows:

  1. The diagram generated in Step 3 is displayed in the Viewer window (bottom right panel). It can be exported by clicking the Export button and selecting Save as Image..., as shown in Figure 6.5.
Screenshot of the RStudio Viewer pane with the Export dropdown open, offering Save as Image, Copy to Clipboard, and Save as Web Page; behind the menu is a node diagram branching from a Project_ID node to folder and file nodes such as Data, Figures and Tables, MS, Output, Reports, README.md, and Scripts leading to the five numbered R scripts.

Figure 6.5: Snapshot showing how to export the diagram in RStudio.

  1. Once you have executed the previous step, a window will open allowing you to select the image format, directory, and file name, as shown in Figure 6.6.
Screenshot of the RStudio Save Plot as Image dialog, with the Image format dropdown open showing PNG highlighted plus JPEG and TIFF, Directory and File name fields on the left, Width and Height with a Maintain aspect ratio checkbox on the right, and a preview of the Project_ID node diagram above the Save and Cancel buttons.

Figure 6.6: Snapshot showing how to save the diagram in RStudio.

To learn more about the options for exporting or saving DiagrammeR graphs, please visit the following website:

https://rich-iannone.github.io/DiagrammeR/io.html

6.3.6.4.3 Now Diagram the Project You Inherited

Run the same procedure on Messy_Project. Set your working directory to Chapter_5, as you did in PART A, and work in a script rather than the console:

### Step 1: list every file in the project you inherited
filesInfo <- file.info(list.files(path = "Messy_Project", recursive = TRUE,
    full.name = TRUE))
### Step 2: convert that data.frame into a data.tree object
messy <- data.tree::as.Node(data.frame(pathString = rownames(filesInfo)))
print(messy)
### Step 3: style and plot it
data.tree::SetGraphStyle(messy$root, rankdir = "LR")
data.tree::SetEdgeStyle(messy$root, arrowhead = "vee", color = "grey",
    penwidth = "2px")
data.tree::SetNodeStyle(messy, style = "rounded", shape = "box")
# NOTE: the Project_ID example above also styled its
# 03_Scripts and R_functions folders. Those two lines are
# deliberately missing here, because this project HAS no
# folders. Asking data.tree to style a node that does not
# exist fails with 'attempt to set an attribute on NULL',
# which is itself informative.
plot(messy)

Figure 6.7: Diagram of the structure of the inherited project. Every file sits at the same level, directly under the project root: there is no structure to show.

Compare Figure 6.7 with Figure 6.4. One has depth: data in one place, outputs in another, code in another, and you can tell what a file is from where it lives. The other is a fan of eight files hanging off a single node.

💡 What the flat diagram is telling you

A diagram with one level is not a diagram of a simple project. It is a diagram of a project with no plan. Nothing in it says which files are inputs, which are outputs, which are code, or which of three scripts to run first. The structure is the cheapest documentation a project can have, and this one has none of it.

Notice also what the diagram cannot show you: it has no way to tell you that two of those files are near-identical copies of a third, or that one of the scripts overwrites its own input. Structure is necessary and not sufficient, which is why the rest of this part is about the code itself.

6.3.6.5 Software Licenses

Code and software are typically licensed under the GNU Affero General Public License. This is a free, copyleft license for software and other types of works, specifically designed to promote community cooperation, especially in the context of network server software.

For example, this is the license used by the instructor for this class.

6.3.6.6 Apply Basic Programming Standards

6.3.6.6.1 Informative, Consistent Naming

Good naming practices should extend to all files, folders, and even objects in your analysis. This helps make the contents and relationships among the elements of your analysis understandable, searchable, and logically organized (see Figure 6.4 for examples, and Chapter 5: Data Management for more details).

6.3.6.6.2 Writing Pseudocode

Pseudocode is an informal, high-level description of how a computer program or algorithm operates. It uses the structural conventions of a standard programming language (in this case, R) but is intended for human understanding rather than execution by a machine.

Here, you will define the major steps in your process (and their associated tasks) and link them to R functions—either existing ones or those you need to create. This serves as the backbone of your code and supports the process of writing it.

6.3.6.6.3 Writing Code

Writing clear, reproducible code has (at least) three major benefits:

  1. It makes returning to the code much easier months later—whether you’re revisiting an old project or making revisions after peer review.
  2. The results of your analysis are easier for others (e.g., readers or reviewers) to examine and validate.
  3. Clean, reproducible code can encourage broader adoption of any new methods you develop.

To support writing effective code, it is recommended to follow the workflow shown in Figure 6.3. The next section explains each part of this workflow and offers tips for writing better code. Although the workflow represents a ‘gold standard,’ even adopting a few of its elements can significantly improve the clarity and quality of your code.

6.3.6.6.4 Style Guides

The foundation of writing readable code is choosing a clear and consistent coding style—and sticking to it. Some key elements to consider when developing your coding style include:

  • Using meaningful file names, and numbering them if they are part of a sequence (see Figure 6.4 for examples, and Chapter 5: Data Management for more details).
### Naming files
# Good
fit-models.R
utility-functions.R

# Bad
foo.r
stuff.r
  • Use concise and descriptive object names. Variable names should typically be nouns, while function names should be verbs. Avoid using names that already exist as variables or functions in R.
### Naming objects
# Good
day_one
day_1

# Bad
first_day_of_the_month
DayOne
dayone
djm1
  • Use spacing to improve readability: add spaces around operators (=, +, -, <-, etc.) and after commas, just as you would in a sentence.
### Spacing
# Good
average <- mean(feet / 12 + inches, na.rm = TRUE)

# Bad
average<-mean(feet/12+inches,na.rm=TRUE)
  • Use two spaces for indentation—avoid tabs, and never mix tabs and spaces.
### Indentation
long_function_name <- function(a = "a long argument", 
                               b = "another argument",
                               c = "another long argument") {
  # As usual code is indented by two spaces.
}
  • Use <- for assignment, not =.
### Assignment
# Good
x <- 5

# Bad
x = 5

The most important role of a style guide, however, is to ensure consistency across scripts.

6.3.6.6.5 Commenting Code

How often have you revisited an old script six months later and struggled to remember what you were doing? Or taken on a script from a collaborator and found it difficult to understand what their code does and why?

An easy way to make code more readable and reproducible is the liberal—and effective—use of comments. A comment is a line of text that is visible in the script but is not executed. In both R and Python, comments begin with a #.

A good principle to follow is to comment the ‘why’ rather than the ‘what’. The code itself shows what is being done; it is far more important to explain the reasoning behind a particular section, or to clarify any nonstandard or complex parts of the code.

It is also good practice to use comments to provide an overview at the beginning of a script and to use commented lines of --- to visually break up the script—for example, in R:

# Load data -------
6.3.6.6.6 Writing Functions

When analyzing data, you often need to repeat the same task multiple times. For example, you might have several files that all require loading and cleaning in the same way, or you might need to perform the same analysis for multiple species or parameters. The best way to handle these tasks is to write functions and store them in a dedicated folder (e.g., R_functions as shown in 6.4). These functions can then be loaded into the R environment—and made available for use—by calling the source() function, which should be placed at the top of your R script. For more details, please see Chapter 1, Part D.

6.3.6.6.7 Testing Scientific Code

In the experimental sciences, rigorous testing ensures that results are accurate, reproducible, and reliable. Testing confirms that the experimental setup functions as intended and helps quantify any systematic biases. Since experimental results are not trusted without such validation, the same standard should apply to your code.

Testing scientific code helps ensure it works as intended and allows you to understand and quantify any limitations. Additionally, using tests can speed up code development by identifying errors early.

While establishing formal testing protocols is especially important when designing R packages, informal testing is also valuable. The instructor recommends that students load their written functions and run ad hoc tests in the command line to verify that the functions perform as expected.

6.3.6.7 Code Portability

When working collaboratively, ensuring code portability between machines is crucial—that is, will your code run correctly on someone else’s computer? Portability is also important when running code on different servers, such as a High Performance Cluster.

One effective way to improve code portability is to avoid using absolute paths and instead use relative paths (see https://en.wikipedia.org/wiki/Path_(computing)#Absolute_and_relative_paths).

  • An absolute path provides the full address to a folder or file.
  • A relative path specifies the location of a file relative to the current working directory. Relative paths are preferred because they enhance code portability.

For example, consider the file species_data.csv stored in the 01_Raw_Data folder of the example project (Figure 6.4):

# Absolute path -----------------------------
C:/Project_ID/01_Raw_Data/species_data.csv

Project_ID = Project root folder = working directory

# Relative path ------
01_Raw_Data/species_data.csv

Relative paths are especially useful when transferring projects between computers. For example, while you might have stored your project folder in C:/Project_ID/, someone else might have theirs in C:/Users/My Documents. Using relative paths—and running your code from the project folder—helps avoid file-not-found errors.

RStudio is specifically designed to facilitate code portability. For instance, you can easily set the working directory in RStudio by clicking:

Session > Set Working Directory > To Source File Location

Alternatively, you can use the setwd() function. For more details, please see Chapter 2.

6.3.6.8 Document and Manage Software Dependencies

Reproducibility also means ensuring that someone else can reuse your code to obtain the same results as you (see Appendix 1 for more details).

To enable others to reproduce the results in your report, you need to provide more than just the code and data. You must also document the exact versions of all packages, libraries, and software used, as well as potentially your operating system and hardware.

R itself is quite stable, and the core development team takes backward compatibility seriously—meaning old code generally works with recent versions of R. However, default values for some functions can change, and new functions are regularly introduced. If you wrote your code on a recent R version and share it with someone using an older version, they might not be able to run it. Similarly, code written for one package version may produce different results with a newer version.

6.3.6.8.1 Reporting R Packages and Versions

In R, the simplest—and a very useful—way to document your dependencies is to include the output of sessionInfo() (or devtools::session_info()). This output shows all the packages and their versions loaded in the session used to run your analysis. Anyone wishing to recreate your work will then know which packages and versions to install. See Appendix 2 for more details.

For example, here is the output of sessionInfo() showing the R version and packages used to create this document:

sessionInfo()
## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Sequoia 15.7.9
## 
## Matrix products: default
## BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
## 
## locale:
## [1] en_US/en_US/en_US/C/en_US/en_US
## 
## time zone: America/Chicago
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] klippy_0.0.0.9500    rticles_0.27         DiagrammeR_1.0.11   
##  [4] DT_0.34.0            data.tree_1.2.0      kfigr_1.2.1         
##  [7] devtools_2.4.6       usethis_3.2.1        bibtex_0.5.1        
## [10] knitcitations_1.0.12 htmltools_0.5.9      prettydoc_0.4.1     
## [13] magrittr_2.0.4       dplyr_1.2.1          kableExtra_1.4.0    
## [16] formattable_0.2.1    bookdown_0.46        rmarkdown_2.30      
## [19] knitr_1.50          
## 
## loaded via a namespace (and not attached):
##  [1] xfun_0.54          bslib_0.9.0        htmlwidgets_1.6.4  remotes_2.5.0     
##  [5] visNetwork_2.1.4   vctrs_0.7.3        tools_4.5.2        crosstalk_1.2.2   
##  [9] generics_0.1.4     tibble_3.3.0       RefManageR_1.4.0   pkgconfig_2.0.3   
## [13] RColorBrewer_1.1-3 assertthat_0.2.1   lifecycle_1.0.5    compiler_4.5.2    
## [17] farver_2.1.2       stringr_1.6.0      textshaping_1.0.4  sass_0.4.10       
## [21] yaml_2.3.12        pillar_1.11.1      jquerylib_0.1.4    ellipsis_0.3.2    
## [25] cachem_1.1.0       magick_2.9.0       sessioninfo_1.2.3  tidyselect_1.2.1  
## [29] digest_0.6.39      stringi_1.8.9      purrr_1.2.2        fastmap_1.2.0     
## [33] cli_3.6.5          pkgbuild_1.4.8     withr_3.0.2        scales_1.4.0      
## [37] backports_1.5.0    lubridate_1.9.4    timechange_0.3.0   httr_1.4.7        
## [41] memoise_2.0.1      evaluate_1.0.5     viridisLite_0.4.2  rlang_1.3.0       
## [45] Rcpp_1.1.0         glue_1.8.0         formatR_1.14       xml2_1.5.1        
## [49] pkgload_1.4.1      svglite_2.2.2      rstudioapi_0.17.1  jsonlite_2.0.0    
## [53] R6_2.6.1           plyr_1.8.9         systemfonts_1.3.1  fs_1.6.6
6.3.6.8.2 Using R Packages to Help Manage Dependencies and Recreate Setup

This topic will not be covered in this class, but it is worth noting that there are at least two R packages designed to better manage dependencies and help recreate your project setup. These packages are:

The instructor encourages students to discuss with their supervisors whether these packages might be useful for their projects.

6.3.7 👥 Group Activity: Read the Code You Inherited (67 minutes, split across two classes)

6.3.7.1 Aim

In PART A you took this project apart and never once opened its code. There are three R scripts in that folder and nobody in this room has read a line of them. The person who wrote them is still not answering email, the manuscript is still half written, and one of the three is called analysis_USE THIS ONE.R.

6.3.7.2 Learning Outcomes

By the end of this activity, students are able to:

6.3.7.3 Requirements

Three things have to be on your laptop before the activity starts, all set up in What to Set Up on Your Computer Before Class.

⚠️ Without these you will spend the activity setting up instead of reading code

  • The project, Chapter_5/Messy_Project/, exactly as it was in PART A. Do not download it again and do not tidy it.
  • Your second script, Chapter_5_reproducible_code.R, open in RStudio.
  • Your diagram of the project structure, from the before-class setup.

You will also want the Defect_log from PART A open beside you: the code faults you are about to find are frequently the cause of the data faults you logged last week.

6.3.7.4 Overview

Five steps over two classes. Steps 1 and 2 are the first class, and Steps 3 to 5 open the second one. Those three steps also close PART A, because the data defects you logged there and the code defects you find here are rows in the same log and belong in the same report.

Class 1

Step What you do Who Time
1 Try to Run It Attempt the script on your own machine and record what happens whole class 7 min
2 Diagnose your area Read the code for the one kind of fault your pair owns your pair 15 min

Step 2 is finished at home, and your rows have to be in your knitted Defect_log before Class 2, because Step 3 is read straight off it.

Class 2

Step What you do Who Time
3 Report What You Found Your data defect and your code defect, read from the same log your pair 20 min
4 Write the Clauses One clause for the data, one for the code your pair 15 min
Total in class 35 min

Step 5 is done at home, and the charters are reviewed at the start of the next class.

Step What you do Who Time
5 Merge Into Your Charter The two pairs in each group reconcile all eight clauses your group, at home 10 min
6.3.7.4.1 Your pair, and the kind of code fault it owns

Same four pairs as PART A, formed inside the same two charter groups (Pairs 1 and 2 in one, Pairs 3 and 4 in the other), and each pair’s new territory is the code counterpart of what it already owns. You know this project’s data better than anyone; now find out what the code did with it.

Pair In PART A you owned Now you own Your reference sections
Pair 1 the names and layout of the files Portability: where the code expects things to be Code Portability, File System Structure
Pair 2 the formats and the values inside the files Correctness: what the code does with values it cannot read as numbers Testing Scientific Code, and defensive programming from Chapter 2
Pair 3 documentation and metadata Readability: whether a human can follow it Naming, Commenting Code, Style Guides, Writing Functions
Pair 4 preservation and sharing Workflow and dependencies: what the code does to its own inputs, and whether it can be re-run at all Principles of a Good Analysis Workflow, Dependencies, Licenses

You are adding to the same Defect_log.csv, and giving Defect_log.Rmd a second section headed # Code defects with its own kable() chunk. Same five columns as before, same standard, and pseudocode for the fix wherever you can write it.

6.3.7.5 Step-by-Step Procedure

Step 1 is the same for everybody, and it is deliberately the first thing you do: you find out whether the thing runs before you have any opinion about how it is written. Step 2 is your pair’s own work.

6.3.7.5.1 Step 1: Try to Run It (whole class, 7 minutes)

Whole class, before any discussion. First, find the code in R rather than taking my word for what is in the folder:

# What code did we inherit? The pattern matches file names
# ending in .R
list.files("Messy_Project", pattern = "\\.R$")
[1] "analysis_USE THIS ONE.R" "analysis.R"
[3] "analysis2.R"

Three scripts for one analysis, and one of them is telling you which to use. That name is the only guidance the author left, and in a moment you will find out what it is worth.

The script named analysis_USE THIS ONE.R is, by its own name, the one that produced the results. Run it:

# From Chapter_5, with Messy_Project/ beside your script
source("Messy_Project/analysis_USE THIS ONE.R")
Error in setwd("/Users/jdoe/Desktop/stuff/Messy_Project") : 
  cannot change working directory

It fails on line 1. Not on line 40, not on some subtle numerical difference: on the very first line, on every machine in this room, and it would have failed the same way for a reviewer, a journal, or the author’s own next laptop.

Three questions, as a class:

  1. How far did it get? Line 1 of 8.
  2. Is analysis.R any better? It has no setwd() at all. Run it from Chapter_5 and say what happens, and why that is a different failure with the same cause.
  3. What could “this analysis is reproducible” possibly mean for a script that cannot start?

💡 This is the cheapest defect in the chapter, and the most common

setwd(“/Users/jdoe/Desktop/stuff/Messy_Project”) is an absolute path. For that line to work, a computer must have a user called jdoe, with a Desktop, containing a folder called stuff, containing this project. Exactly one machine in the world ever satisfied that, and it is gone.

The fix costs nothing and takes one line: delete it, and run the script from the project root using relative paths. That is the whole of code portability, and it is worth noticing that this project’s single most destructive defect is also its easiest to have avoided.

6.3.7.5.2 Step 2: Diagnose the Area Your Pair Owns (your pair, 15 minutes)

👥 Go straight to your own section

You need one of the four. If you are unsure which pair you are, the table above says.

All three scripts are short, so read all three, not just the one with your defect in it. Record what you find in Defect_log.csv, and remember that you are diagnosing, not repairing: the fix goes in the log as pseudocode.

6.3.7.5.2.1 Pair 1: Portability
  • Two of the three scripts begin with setwd(). Write the path down. Then list everything that must be true of a computer for that line to succeed.
  • Make it portable in pseudocode: what should the first two lines of a script in this project be instead, so that it runs for anybody who has the project folder?
  • analysis.R has no setwd(). Does that make it portable? Run it from Chapter_5 and explain the error. Then say what working directory it silently assumes.
  • Which RStudio feature removes the need for setwd() altogether? Name it, and say why it is better than a correct setwd().
  • Rebuild this project on paper. Your model is the worked example: read Numbering the Flow, look at Figure 6.4, and if you want the reasoning for each directory, open Project_ID on GitHub and read its README.md. Then write out the numbered directories you would create, and place each of the eight inherited files into one of them. Two cases will give you trouble, and they are the interesting ones:
    • At least one file does not belong anywhere in a shared project folder. Identify it, say why, and say where it should live instead. (You logged this in PART A.)
    • Three scripts, no order. Which runs first? Give them numbered names in the style of Figure 6.4, and say what each should be responsible for. This is the job you did for the data files in PART A, now for code.
  • Then say which path each script should use to reach the data under your new structure, and confirm that no script writes backwards into 01_Raw_Data/.
6.3.7.5.2.2 Pair 2: Correctness

In PART A you found what is wrong with the mass column. Now read the line that consumes it.

d <- read.csv("Messy_Project/copy of data(1).csv")
d$mass <- as.numeric(d$mass)  #the line in all three scripts
sum(!is.na(d$mass))  #how many survived?
mean(d$mass, na.rm = TRUE)  #write this number down
  • How many of the twelve measurements survive that one line? What does R say while destroying the others, and how easy would that be to miss in a script you ran a month ago?
  • Compute the mean and write it down. Then explain the number to somebody who has not seen the data. Is it a possible body mass?
  • lm(mass ~ trt + temp, data = d) is the model. What class is temp? What does lm() do with a character column, and what are the resulting coefficient names?
  • How many parameters does that model estimate, and from how many usable rows? Report the residual degrees of freedom.
  • Write, in one line, the test that should have stopped this script before it produced any number at all. Compare your answer to the guard you wrote in Chapter 2 - PART E.
6.3.7.5.2.3 Pair 3: Readability
  • Count the comments across all three scripts. Quote the only one that attempts to explain anything, and say what its wording tells you about how confident the author was.
  • The objects are called d and m. Judge those against Informative, Consistent Naming and propose better ones. Which should be nouns?
  • All three scripts begin with the same lines. What does Writing Functions say to do about repetition like that, and where should the result live?
  • Check the three scripts against the style guide: spacing, assignment operator, indentation. List what fails.
  • Rewrite the first ten lines of one script as they should have been: what does a reader need to know before they reach any code? You are writing a header, not a comment on every line.
6.3.7.5.2.4 Pair 4: Workflow and Dependencies
  • Read the last line of analysis_USE THIS ONE.R again. You logged it in PART A as a data defect. Now state it as a workflow defect: what does a script that writes to the file it reads do to the possibility of ever re-running it?
  • All three scripts read copy of data(1).csv, which you established in PART A is not the master copy. What does that mean for every number this project has produced?
  • Nothing anywhere records which package versions were used. Name the one function from Document and Manage Software Dependencies that fixes this, and say exactly where in a script it belongs.
  • There is no LICENSE file. What are you, the inheritor, actually permitted to do with this code? See Software Licenses.

6.3.7.6 Reporting

Reporting opens the second class of this part, and it closes PART A as well. Your pair reports two findings out of one log: the data defect you diagnosed in the data management class, and the code defect you diagnosed in Step 2. Bring your script, your knitted Defect_log and your group’s charter.

💡 Why the data half waited for this class

You have had a few days with the log. Some of what looked like a catastrophe on the day turns out to be routine, and some of what you passed over turns out to matter more than you thought. A diagnosis you can still defend after sleeping on it is worth more than one delivered while the folder is still open in front of you.

The second reason is in your log now. The code faults you found in Step 2 are frequently the cause of the data faults you logged in PART A, and reporting the two together is the only way that shows.

6.3.7.6.1 Step 3: Report What You Found (your pair, 20 minutes)

Three minutes per pair, read straight off your knitted log, and cover both of your areas:

  1. Your worst data defect, from PART A: name the file, quote the evidence, say whether it can still be repaired, and read out your pseudocode for the repair.
  2. Your worst code defect, from today, on the same four points.
  3. The practice that would have prevented each of them. Those two answers are your two clauses in Step 4, so say them in a form you can write down.

Read the pseudocode out. It is the part the rest of the class learns most from, because it is where somebody else’s defect becomes a procedure you could follow, and because the steps that make people argue are exactly the ones where the data cannot settle the question.

A defect you cannot fix still counts, and usually counts for more. Nobody is expected to arrive with a solution. The question this step answers is what is wrong with this project, and a pair that says “this column imports as text, four values are unusable, and we do not yet know how to repair them” has done the job completely.

Listen to all four reports. You adopt all four pairs’ conclusions in Step 5, not only your own.

💡 The number that should have stopped everything

Pair 2 will have it. Run as written, with those values left in and the coercion unchecked, this project’s analysis reports a mean body mass of −133.87. Negative. Nobody noticed, and the script’s own comment says the paper’s numbers came from it.

It gets worse on inspection. Only 7 of the 12 measurements survived the coercion. temp was never converted, so lm() silently treated it as a category with seven levels, one of which is 22±0.5. The model estimates six parameters and is left with one residual degree of freedom.

None of this produced an error. One warning, easily scrolled past, and then a table of coefficients that looks exactly like a result. This is what the chapter means by trusting code: not “did it run” but “do I have any reason to believe the number”.

When all four pairs have reported, we return as a class to the stage nobody was assigned:

💡 Count how many of these defects are Planning failures

Go through the eight findings and ask, for each: when could this have been prevented at no cost? Nearly every answer is “before any data existed, and before a line of code was written”. A naming convention costs nothing to decide and a fortune to retrofit. A data dictionary written during digitisation takes an hour; reconstructed afterwards it is guesswork, and for the treatments in this project it is not possible at all. A relative path costs one line at the start and cannot be retrofitted into somebody else’s dead laptop.

This is why Plan sits at the hub of Figure 6.2 rather than at the start of the ring, and it is why a data management plan is a research instrument rather than paperwork for your funder.

6.3.7.6.2 Step 4: Write the Clauses (your pair, 15 minutes)

Work in your pair, and write two clauses from your own two findings. All four pairs do this at the same time, so there are eight clauses in the room when the step ends. You will see the other six in Step 5.

One clause from each of your two findings:

  • from the data defect you diagnosed in PART A, into the Data lead section of your group’s charter, which owns protocols, notes, file naming, storage, backup and metadata
  • from the code defect you diagnosed in Step 2, into the Analysis lead section, which owns “code and statistics: the analysis plan, what is decided before the data are seen, and how an analysis is made re-runnable”

A clause is not a description of your defect. It is the standard that would have prevented it. You already said the middle column out loud in Step 3, when you named the practice behind the fault:

What your pair found The practice behind it The clause you write
setwd("/Users/jdoe/Desktop/stuff/Messy_Project") on line 1 of two scripts code has to run wherever the project is, not where its author kept it No script contains setwd(). Every script runs from the project root and uses relative paths only.

Notice what survives the generalisation and what does not. The clause does not mention jdoe, this project, or even this defect: somebody joining your group next year can act on it without ever seeing Messy_Project. That is the test of whether you have written a standard rather than a complaint.

The charter is in our shared Drive at Reproducible_Science > Chapters > Chapter_1 > Part_B. Hold both clauses to the charter’s own test, from Chapter 1:

A charter states what this group will always do. Each commitment is specific enough that somebody else can tell whether it was kept.

So “we will name our files sensibly” is not a clause, and neither is “we will write cleaner code”. Both fail the test, because nobody can tell whether they were kept. These pass.

For the data:

  • Raw data files are written once and never edited. All cleaning happens in a script that reads from Data/raw/ and writes to Data/processed/.
  • Every dataset is accompanied by a README.md defining each column, its unit, and its missing-value code, written before analysis begins.
  • No dataset leaves the group until a named author has checked it for personal information and for coordinates of listed species.

For the code:

  • No script writes to a file that it reads.
  • Any coercion of a column is immediately followed by a check that the number of missing values has not changed, and the script stops if it has.
  • Every analysis script ends by recording sessionInfo() in the output.

Type both into the Clauses sheet before the step ends, on your pair’s row. Three cells: what you found, a line each on the data defect and the code defect, then the data clause and the code clause beside them, so that anybody reading the sheet can see whether the clause actually follows from the finding. It lives beside the charters in Reproducible_Science > Chapters > Chapter_5, where your knitted Defect_log also goes, in your pair’s subfolder from PART A.

This is not administration. Six of the eight clauses will be written into a charter by people who were not in your pair, and they cannot insert what they cannot read. A clause dictated across the room becomes four slightly different clauses in two charters; a clause written down once becomes the same clause in both.

If you want more worked examples, there are charter clause templates in Chapter 1. And note one outcome worth hoping for: if your group’s charter already covered the defect you found, say so in your report. That is the best possible result of this exercise, and it means your charter was ambitious enough.

6.3.7.6.3 Step 5: Merge Into Your Charter (your group, 10 minutes, at home)

This step is homework, and it is the last thing Chapter 5 asks of you. Your charter is a live document, so the group does not need to be in one room to do it, and the clauses are already written: they are in the sheet. We go over the charters briefly at the start of the next class.

There are two charters and four pairs, so the clauses have to travel. The mechanism is:

  1. Split into your two charter groups. Each group is the two pairs that formed inside it: Pairs 1 and 2, or Pairs 3 and 4.
  2. Open your charter, which is a live document in our shared Drive at Reproducible_Science > Chapters > Chapter_1 > Part_B. You are revising it, not starting a new one.
  3. Open the Clauses sheet and insert all eight: the four data clauses into the Data lead section of your charter and the four code clauses into the Analysis lead section. Copy the wording, do not retype it, so that the pair who wrote a clause and the charter that adopts it say the same thing.
  4. Reconcile the wording with what your charter already says, and where a new clause contradicts an existing one, resolve it now rather than keeping both.

💡 Both charters should end up with the same eight commitments

Differently worded, perhaps, and sitting in a different order, but the same eight: four under Data lead and four under Analysis lead. If one charter quietly ends up with six, the class has lost a finding and the pair that found it has no way of knowing. Before you leave, count them.

Your charter now has two of its three domains tested against a real project. PART A worked the Data lead section, today worked the Analysis lead section, and the Reporting lead is next, in Chapter 6, where the question becomes what leaves the group and what a reviewer is entitled to ask for.

Chapter 1 promised you this:

At the end of the semester you should be able to return to your charter and find that you now know how to implement what you committed to today, and to judge whether you were ambitious enough.

This is the first time that promise comes due. Data management and reproducible code were two of the three domains your charter claimed to cover, and you have just spent two classes finding out what it costs when they are absent.

6.3.7.6.4 What you leave with, and where it goes

Two threads leave this room.

Your pseudocode joins the data pseudocode from PART A as the task list for your own individual project, with Chapter 7 as the reference for how: paths that work anywhere, a script that reads raw data and never writes over it, and coercion that fails loudly instead of quietly.

And in Chapter 6 the roles reverse. You have just spent an hour as the person who inherits somebody’s code and has to decide whether to trust it. That is exactly what a peer reviewer does, and you now know how little an error-free run proves.

7 Chapt. 6 – Getting Published and Peer Review

7.1 Introduction

This chapter is subdivided into two parts as follows:

7.2 Class Outline

Only the licences are taught in class. The rest of Chapter 6 is read at home. The licence block opens the work session that follows the Chapter 5 PART B classes, together with a short look at the charters those classes produced.

In class — the licences

Block Time Who
A short pass over the clauses you wrote into your charters in Step 5 5 min Students, in 2 groups
Open Access and Creative Commons Licenses, carried over from Chapter 4, with Software Licenses from Chapter 5 beside them 15 min Instructor
Total in class 20 min

The rest of that class is a work session on your bioinformatics tutorials.

Read at home

Block Time Who
PART A: Getting published: the four steps, from selecting a journal to acceptance, with the slides in Teaching Material 30 min Students, at home
PART B: Writing papers in R Markdown, with the slides in Teaching Material 20 min Students, at home

Why this part of the chapter is reading rather than class time. It is a sequence of procedures: choose a journal, write to its requirements, submit, answer the reviewers. None of it needs a room, and both decks are embedded in the page, so reading it loses nothing. Bring your questions to any class after you have read it.

Open Access and Creative Commons land in the right place here. They were written for Chapter 4, but choosing where to publish is a Chapter 6 question: selecting your journal includes deciding whether it is open access, what that costs, and what licence your work carries once it is out.

And a paper is not the only thing you license. Submitting one means licensing three different things, under three different instruments, which is why Software Licenses is taught beside Creative Commons rather than left in Chapter 5:

What you license Under what Where it is covered
the article a Creative Commons licence, usually chosen for you by the journal’s open access route Chapter 4
the data a licence named in the deposit, where you also state rights, licences and permissions Chapter 5 PART A
the code a software licence, which is a different kind of instrument and absent by default Chapter 5 PART B

7.3 Getting published

7.3.1 Aim

This part of Chapter 6 aims to introduce the procedures used to present research and disseminate it through publications.

To support that aim, we will focus on discussing:

  1. The importance of knowing the key conclusion that you want to communicate and revolve your publication around it.
  2. The steps towards getting your research published:
    • Selecting your journal.
    • Writing your manuscript.
    • Submitting your manuscript.
    • Acceptance and publication.

7.3.2 Resources

7.3.2.1 Teaching Material

The presentation associated with this class is available here:

💡 Best browser to view presentation

  • For optimal viewing of this presentation in full-screen presentation mode, the instructor recommends using Mozilla Firefox. Its built-in PDF viewer supports seamless full-screen display with easy navigation tools, making it ideal for presentations without the need for external plugins. Other browsers may not handle PDF presentation features as reliably.

7.3.2.2 Literature and Web Resources

This part is mostly based on the following books, publications and web resources:

Books and Guides

Publications

  • Allen et al. (2014) – Comment published in Nature advocating for a “taxonomy” of author roles.
  • Fox and Burns (2015) – A publication investigating the relationship between manuscript title structure and success in Ecology.

Websites

7.3.3 Why publish?

Publishing research results is the one thing that unites scientists across disciplines, and it is a necessary part of the scientific process. You can have the best ideas in the world, but if you don’t communicate them clearly enough to be published, your work won’t be acknowledged by the scientific community.

By publishing you are achieving three key goals for yourself and the larger scientific endeavor:

  • Disseminating your research.
  • Advancing your career.
  • Advancing science.

In biology, publishing research is equal to journal articles.

7.3.4 Know your message

Before beginning writing your journal article and thinking where to submit it, it is important to thoroughly understand your own research and know the key conclusion you want to communicate (see chapter 3). In other words, what is your take home message?

Consider your conclusion and ask yourself, is it:

  • New and interesting?
  • Contributing to a hot topic?
  • Providing solutions to difficult problems?

If you can answer ‘yes’ to all three, you have a good foundation message for a paper.

Shape the whole narrative of your paper around this message.

7.3.5 Steps towards getting your research published

Once you know your message, getting your research published will be a four steps process:

  • Step 1: Selecting your journal.
  • Step 2: Writing your manuscript.
  • Step 3: Submitting your manuscript.
  • Step 4: Acceptance and publication.

Each step will be discussed below. Please seek support from your supervisor to learn more about specifics from you field.

7.3.6 Step 1: Selecting your journal

To target the best journal to publish your research, you need to ask yourself what audience do I want my paper to reach?

Your manuscript should be tailored to the journal you want to submit to in terms of content and in terms of style (as outlined in journals’ author guidelines). To confirm that a journal is the best outlet to publish your research ask yourself this question: can I relate my research to other papers published in this journal?

Here are some things to consider when choosing which journal to submit to:

7.3.6.1 Journal aims and scope

Look closely at what the journal publishes; manuscripts are often rejected on the basis that they would be more suitable for another journal. There can be crossover between different journals’ aims and scope – differences may be subtle, but all important when it comes to getting accepted.

Do you want your article read by a more specialist audience working on closely related topics to yours, or researchers within your broader discipline?

Once you have decided which journal you are most interested in, make sure that you tailor the article according to its aims and scope.

7.3.6.2 Editors and editorial boards

It is a good sign if you recognize the names of the editors and editorial board members of a journal from the work you have already encountered (even better if they contributed to some of the references cited in your manuscript). Research who would likely deal with your paper if you submitted to a journal and find someone who would appreciate reading your paper. You can suggest handling editors in your cover letter or in the submission form, if it allows, but be aware that journals do not have to follow your suggestions and/or requests.

7.3.6.3 Impact factor and other metrics

A summary of our previously discussed material is presented below to provide more context for this chapter, but please consult Chapter 4 for more details on this topic.

Impact factors are the one unambiguous measure widely used to compare journal quality based on citations the journal receives. However, other metrics are becoming more common, e.g. altmetric score measuring the impact of individual articles through online activity (shares on different social media platforms etc.), or article download figures listed next to the published paper.

None of the metrics described here are an exact measure of the quality of the journal or published research. You will have to decide which of these metrics (if any) matter most to your work or your funders and institutions.

7.3.6.4 Open access

Do you need to publish open access (OA)? Some funders mandate it and grant money often has an amount earmarked to cover the article processing charge (APC) required for Gold OA. Some universities have established agreements with publishers whereby their staff get discounts on APCs when publishing in certain journals (or even a quota of manuscripts that can be published for “free” on a yearly basis). If you do not have grant funding, check whether your university or department has got an OA fund that you could tap into.

However, if you are not mandated to publish OA by your funder and/or you do not have the funds to do so, your paper will still reach your target audience if you select the right journal for your paper. Remember, you can share your paper over email.

7.3.6.5 Author guidelines

Author guidelines will outline the journal’s requirements for submissions:

  • Aims and scope.
  • Formatting requirements (incl. words limit, numbers of figures, tables, references).
  • Journal policies (e.g. on data sharing and citation).

Always follow the author guidelines, stick to the word limit and tailor your manuscript accordingly. Remember that papers can be rejected immediately if they do not meet the author guidelines.

7.3.6.6 Time to publication

The length of time a paper takes to be peer reviewed does not correlate to the quality of peer review, but rather reflects the resources a journal has to manage the process (e.g. do they have paid editorial staff or is it managed by full-time academics?).

Journals usually give their average time to a decision on their website, so take note of this if time is a consideration for you.

Some journals also make it clear that they are reviewing for soundness of science rather than novelty and will therefore often have a faster review process (e.g. PLoS ONE).

7.3.6.7 Ethics

Ethics can be divided into two groups:

  • Research ethics: this term includes aspects such as how you manage sensitive species information, whether you adhere to animal welfare guidelines and regulations or how you deal with data protection.
  • Publication ethics: this term concerns practices around the publication process. Standards set across scholarly publishing help define good practice and identify cases of misconduct. The Committee on Publication Ethics (COPE) provides the main forum for advice on ethics within scholarly publishing and has issued several sets of guidelines that help journals, editors and publishers handle cases of misconduct such as data fabrication, peer review fraud, plagiarism, etc.

As an author, it helps if you are familiar with what constitutes good practices and what is considered unacceptable. Please see section “Used literature & web resources” for more details on this topic.

7.3.6.8 Authorship

Start talking about authorship and author order for your paper with collaborators at an early stage – before submitting and ideally before writing the paper. To deal with potential issues, ask yourself the question: Who will do what?

Some journals are now encouraging ‘authorship contribution statements’ so check the journal guidelines to see if this is required and how to format it. To help you figuring out authorship associated with your research article, you could apply the CRediT (Contribution Roles Taxonomy) model presented by Wiley. The classification is as follows:

CRediT Classification

  • Conceptualization: Ideas; formulation or evolution of overarching research goals and aims.
  • Data Curation: Management activities to annotate (produce metadata), scrub data and maintain research data (including software code, where it is necessary for interpreting the data itself) for initial use and later reuse.
  • Formal Analysis: Application of statistical, mathematical, computational, or other formal techniques to analyze or synthesize study data.
  • Funding Acquisition: Acquisition of the financial support for the project leading to this publication.
  • Investigation: Conducting a research and investigation process, specifically performing the experiments, or data/evidence collection.
  • Methodology: Development or design of methodology; creation of models.
  • Project Administration: Management and coordination responsibility for the research activity planning and execution.
  • Resources: Provision of study materials, reagents, materials, patients, laboratory samples, animals, instrumentation, computing resources, or other analysis tools.
  • Software: Programming, software development; designing computer programs; implementation of the computer code and supporting algorithms; testing of existing code components.
  • Supervision: Oversight and leadership responsibility for the research activity planning and execution, including mentorship external to the core team.
  • Validation: Verification, whether as a part of the activity or separate, of the overall replication/reproducibility of results/experiments and other research outputs.
  • Visualization: Preparation, creation and/or presentation of the published work, specifically visualization/data presentation.
  • Writing – Original Draft Preparation: Creation and/or presentation of the published work, specifically writing the initial draft (including substantive translation).
  • Writing – Review & Editing: Preparation, creation and/or presentation of the published work by those from the original research group, specifically critical review, commentary or revision – including pre- or post-publication stages.

Here is an example of an author contributions paragraph inspired by the CRediT model:

F.F., N.A.B. and S.B. designed study; F.F., E.B., S.P.B., J.M., and S.B. compiled data; F.F., S.B. and J.M. performed analyses; S.I.B., P.M.H., A.L., D.P.L., S.M., H.R., C.R., D.W.S., and P.T. provided material and/or sequences; F.F., J.M. and S.B. wrote the manuscript; all authors critically read and revised the manuscript, and approved the final submitted version.

7.3.7 Step 2: Writing your manuscript

7.3.7.1 Planning to write

Develop a narrative that leads to your main conclusion and develop a backbone around that narrative. The narrative should progress logically, which does not necessarily mean chronologically. Work out approximate word counts for each section to help manage the article structure and keep you on track for word limits.

It is important to set aside enough time to write your manuscript and – importantly – enough time to edit, which may actually take longer than the writing itself.

7.3.7.2 Structure

The article structure will be defined in the author guidelines, but if the journal’s guidelines permit it, there may be scope to use your own subheadings. By breaking down your manuscript into smaller sections, you will be communicating your message in a much more digestible form.

Use subheadings to shape your narrative and write each subheading in statement form (e.g. ecological variables do not predict genome size variation).

7.3.7.3 Title

The title is the most visible part of your paper and it should thus clearly communicates your key message. Pre-publication, reviewers base their decision on whether to review a paper on the quality of the title and abstract. Post-publication, if you publish in a subscription journal and not OA, the title and abstract are the only freely available parts of your paper, which will turn up in search engines and thus reach the widest audience. A good title will help you get citations and may even be picked up by the press.

Draft a title before you write your manuscript to help focusing your paper. The title needs to be informative and interesting to make it stand out to reviewers and subsequently readers. Some key tips for a successful title include:

  • Write it in statement form. When scanning papers, most people skip to the last sentence of the abstract to look for the key message, so make that sentence your title.
  • Keep it around 15 words – any longer or shorter and it has more chance of being rejected at peer review.
  • Use punctuation to split the main message and qualifier/subtitle e.g. ‘Feeding evolution of a herbivore influences an arthropod community through plants: implications for plant-mediated eco-evolutionary feedback loop’.
  • Keep it general – readers prefer titles that emphasize broader conceptual or comparative issues, and these titles fare better both pre- and post-publication than papers with organism-specific titles. Try to avoid using species names, put them in the abstract and keywords instead.
  • Do not use abbreviations even if they are familiar in your field. You should keep a broad audience in mind.
  • Do not use phrases such as ‘The effect of…’, ‘The involvement of…’. These phrases give the reader scope to question your message – instead tell the reader what they are being told.

7.3.7.4 Abstract

Write your abstract after you have written your paper, when you are fully aware of the narrative of your paper. After the title, the abstract is the most read part of your paper. Abstracts are freely available and affect how discoverable your article is via search engines.

Given its importance, your abstract should:

  • Articulate your new and interesting key message.
  • Outline the methods and results.
  • Contextualize the work.
  • Highlight how your research contributes to the field and its future implications.
  • Have the last sentence communicating the key message.

7.3.7.5 Writing style

Writing with clarity, simplicity and accuracy takes practice and we can all get carried away with what we think is ‘academic writing’ (i.e. long words and jargon) but good science speaks for itself. Write short sentences (ca. 12 words on average).

Every extra word you write is another word for a reviewer to disagree with. Single out the narrative that leads to your main conclusion and write that – it is easy to get side tracked with lots of interesting avenues distracting from your work, but by including those in your paper, you are inviting more criticism from reviewers.

Write in an active, positive voice (e.g. ‘we found this…’ ‘we did this…’) and be direct so that your message is clear. Ambiguous writing is another invitation for reviewers to disagree with you.

In your introduction, state that your research is timely, important, and why. Begin each section with that section’s key message and end each section with that message again plus further implications. This will place your work in the broader context that high-quality journals like.

Draft and redraft your work to ensure it flows well and your message is clear and focused throughout. While doing this process, keep the reader in mind at all times (to have a critical look on your research and its presentation).

7.3.7.6 Keywords

Keywords are used by readers to discover your paper. You will increase chances of your paper being discovered through search engines by using them strategically throughout your paper – this is search engine optimization (SEO).

Think of the words you would search for to bring up your paper in a Google search. Try it and see what comes up – are there papers that cover similar research to your own?

Build up a list of 15–20 terms relevant to your paper and divide them into two groups:

  • A core group of around 5 keywords,
  • A larger group of secondary keywords.

Place your core keywords in the title, abstract and subheadings, and the secondary keywords throughout the text and in figures and tables. Repeat keywords in the abstract and text naturally.

7.3.7.7 References

Reference all sources and do it as you go along (e.g. copy the BibTeX citation into a reference file; see chapter 1 part B), then tidy them once the paper is complete.

Make sure that most of your references are recent to demonstrate both that you have a good understanding of current literature, and that your research is relevant to current topics.

7.3.7.8 Figures and tables

Figures and tables enhance your paper by communicating results or data concisely (more on this topic in chapters 9 and 10).

Use figures and tables to maintain the flow of your narrative – e.g. instead of trying to describe patterns in your results, create a figure and say ‘see Fig. 1’. Not only does this keep your word count down, but a well-designed figure can replace 1000 words!

Figures are useful for communicating overall trends and shapes, allowing simple comparisons between fewer elements. Tables should be used to display precise data values that require comparisons between many different elements.

Figure captions and table titles should explain what is presented and highlight the key message of this part of your narrative – the figure/table and its caption/title should be understandable in isolation from the rest of your manuscript.

Check the journal’s author guidelines for details on table and figure formatting, appropriate file types, number of tables and figures allowed and any other specifications that may apply. Material presented in chapter 1 part C can help you produce figures meeting journal expectations.

7.3.7.9 Editing

Once you have finished writing your manuscript, put it on ice for a week so you come back to it with fresh eyes. Take your time to read it through. Editing can take more time than you expect, but this is your opportunity to fine-tune and submit the best paper possible. Don’t hesitate to seek support from your thesis committee to fasten and streamline this process.

Key things to look out for when editing include:

  • Spelling and grammar – a surprising amount of errors slip through. If you are a non-native English speaker, ask a native speaker, ideally a colleague who knows a little bit about the subject, to read it through, or use a language-editing service if you have the funds to do so.
  • Make sure all statements and assumptions are explained.
  • Remove redundant words or phrases – keep it concise and jargon-free to avoid diluting your message.
  • Abbreviations – check that they have been expanded on the first use.
  • Acknowledgements – make sure all funders are clearly mentioned and that all people who contributed in any way are acknowledged.
  • Keywords – they should be consistent, evenly spaced throughout the text and placed at key points in your manuscript e.g. subheadings.
  • Make sure you have specifically dealt with the hypothesis set out in the introduction – you’d be surprised at the number of papers submitted that don’t!
  • Circulate the manuscript to your co-authors to get their comments and final approval before submission.

7.3.8 Step 3: Submitting your manuscript

You are now ready to submit your paper to your chosen journal. Each journal will have a different submission procedure that you will have to adhere to, and most manage their submissions through online submission systems (e.g. ScholarOne Manuscripts).

Only submit your paper for consideration to one journal at a time otherwise you will be breaching publishing ethics.

7.3.8.1 Cover letters

A great cover letter can set the stage towards convincing editors to send your paper for review. Write a concise and engaging letter addressed to the editor-in-chief, who may not be an expert in your field or sub-field.

The following points should be covered in your cover letter:

  • State your key message and why your paper is important and relevant to the journal.
  • State that your paper is not under review in another journal and hasn’t been published before (although you will most likely adhere to that during the submission process and if it is the case you then don’t need to mention this in the cover letter).
  • The cover letter should be shorter than your abstract and be written in less technical language.
  • Use it to recommend reviewers (include their emails) and/or a relevant handling editor. Pick suggested reviewers with a good reputation to demonstrate both your knowledge of the field and your belief that your paper can stand up to their scrutiny.
  • State any potential conflict of interest with other teams and blacklist potential reviewers.

7.3.8.2 Handling revisions

Very rarely is a paper immediately accepted – almost all papers go through few rounds of reviews before they get published.

If a decision comes back asking for revisions you should reply to all comments politely. Here are some tips on handling reviewer comments and revising your paper:

  • Look at the reviewer comments with scrutiny and make a list of all the points that need to be addressed.
  • Start with the minor revisions such as spelling, grammar, inconsistencies – these are often the most numerous but the easiest to correct.
  • If you disagree with certain comments, disagree politely and with evidence. Do not skip over them when writing your reply.
  • If things can’t be dealt with in this paper then explain that to the editor – reviewers may try to push their own agenda e.g. ‘why don’t you write this paper instead’, but you have the right to disagree if you don’t feel it is appropriate to deal with this in your paper.
  • Respond to comments as thoroughly as you can.
  • Include a point-by-point response to the reviewer comments in the relevant section of the online system.

7.3.8.3 Handling rejection

Reviewers are volunteers, but the service they provide is invaluable – by undergoing peer review, regardless of the outcome, you are receiving some of the best advice from leading experts for free. With this in mind, any feedback you get will be constructive in the end and will lead you on the way to a successful publishing portfolio.

Keep in mind that feedback is another person’s opinion on what you have done, not on who you are, and it is up to you to decide what to do with it.

If your paper is rejected look at the reviewer’s comments and use their feedback to improve your paper before resubmitting it.

7.3.8.4 Appeals

If you are unhappy with a reject decision, 99.9% of the time, move on. However, don’t be afraid of appealing if you have well-founded concerns or think that the reviewers have done a bad job. There are instances where journals grant your appeal and allow you to revise your paper, but in the large majority of cases, the decision to reject will be upheld.

7.3.9 Step 4: Acceptance and publication

Congratulations! By now you should have an acceptance email from the editor-in-chief in your inbox. The process from here will vary according to each journal, but the post-acceptance workflow is usually as follows:

  • Your paper will be published online, unedited, but citable as an ‘Accepted Article’ within a week of acceptance (usually a DOI is assigned and your paper is citable at this stage).
  • Your paper will be copy-edited. The level of copy-editing your paper will receive will vary according to each journal, so it is worth checking your proof thoroughly.
  • Your paper will be typeset and a proof will be sent to you for checking. Author queries will be marked on the proof. At this stage, only minor corrections related to the typesetting are allowed.
  • Your finalized proof will be published online in ‘Early View’.
  • Finally, according to the journal’s schedule, your paper will be placed in an issue (or not if it is an online only journal, e.g. Scientific Reports).

It might be then time to coordinate the publication of a press release or post the link of your article on social media to share your joy!

7.4 Writing papers in R Markdown

7.4.1 Aim

This part of Chapter 6 aims to show how a journal submission can be produced from R Markdown, using templates that generate the LaTeX files journals require.

7.4.2 Resources

7.4.2.1 Teaching Material

The presentation associated with this class is available here:

💡 Best browser to view presentation

  • For optimal viewing of this presentation in full-screen presentation mode, the instructor recommends using Mozilla Firefox. Its built-in PDF viewer supports seamless full-screen display with easy navigation tools, making it ideal for presentations without the need for external plugins. Other browsers may not handle PDF presentation features as reliably.

7.4.2.2 Literature and Web Resources

This part is mostly based on the following books and web resources:

7.4.2.3 Software Requirements

To apply the approach described below make sure that you have a TeX distribution installed on your computer. More information on this topic is available in Install R Markdown Software (Chapter 2). You will also need to install the R rticles package as demonstrated in The rticles package.

7.4.3 Challenges to writing publications in R Markdown

Traditionally, journals are accepting manuscripts submitted in either Word (.doc) or LaTex (.tex) formats. In addition, most journals are requesting figures to be submitted as separate files (in e.g. .tiff or .eps formats). Online submission platforms are collating your different files to produce a .pdf document, which is shared with reviewers for evaluation. In this context, although the .Rmd format is growing in popularity (due to its ability to “mesh” data analyses with data communication), this format is technically currently not accepted by journals. In this document, we are discussing ways that have been developed to circumvent this issue and allow using the approach implemented in R Markdown for journal submissions.

7.4.4 Solution: Develop templates producing LaTex files matching journal requirements!

As mentioned above, many journals support the LaTeX format (.tex) for manuscript submissions. While you can convert R Markdown (.Rmd) to LaTeX, different journals have different typesetting requirements and LaTeX styles. The solution is to develop scripts converting R Markdown files into LaTex files, which are meeting journal requirements.

7.4.5 The rticles package

Submitting scientific manuscripts written in R Markdown is still challenging; however the R rticles package was designed to simplify the creation of documents that conform to submission standards for academic journals (see Allaire et al., 2024). The package provides a suite of custom R Markdown LaTeX formats and templates for the following journals/publishers that are relevant to the EEB program:

  • Biometrics articles
  • Elsevier journal submissions
  • Frontiers articles
  • MDPI journal submissions
  • PeerJ articles
  • PNAS articles
  • Royal Society Open Science journal submissions
  • Sage journal submissions
  • Springer journal submissions
  • The R Journal articles
  • Taylor & Francis articles

An understanding of LaTeX is recommended, but not essential in order to use this package. R Markdown templates may sometimes inevitably contain LaTeX code, but usually we can use the simpler R Markdown and knitr function to produce elements like figures, tables, and math equations.

7.4.5.1 Install rticles and use templates to write publications

  1. Install the rticles package:
  • Type the above code in the R console:
install.packages("rticles")
  • Or use the RStudio interface to install the package by clicking:
Tools -> Install Packages...

Then, type "rticles" in the prompted window to install package.
  • If you wish to install the development version from GitHub (which often contains new article formats), you can do this (note that this code uses a function from the remotes package, Hester et al., 2019):
remotes::install_github("rstudio/rticles")
  1. Create a new R Markdown document in RStudio:
File -> New File -> R Markdown...
  1. In the New R Markdown window, click on From Template in the left panel and select the journal style that you would like to follow for your article (here PNAS Journal Article; see Figure 7.1). Before pushing the OK button, provide a name for your project and set a location where the project will be saved (see Figure 7.1).
Screenshot of the RStudio New R Markdown dialog with the From Template option selected in the left sidebar and a scrollable list of journal templates on the right, where PNAS Journal Article from the rticles package is highlighted, and Name and Location fields below.

Figure 7.1: R Markdown window allowing to select templates following journal styles.

  1. Once you completed this task, a folder will be created (and saved in the path that you provided) containing files associated with the article submission process and the template will be automatically opened in RStudio (see Figure 7.2).
Screenshot of the RStudio editor on the left showing the Submission_PNAS.Rmd tab open at its YAML header, alongside a macOS Finder window on the right listing the project contents including _site.yml, AmJBot.csl, a bibliography .bib file, Figures, index.html, index.Rmd, README.md, site_libs, and a highlighted Submission_PNAS folder.

Figure 7.2: Snapshot showing the template R Markdown and the associated folder created to generate your submission.

  1. In this example, the following suite of files were created (see Figure 7.3):
  • Submission_PNAS.Rmd: R Markdown file that will be used to write your article.
  • pnas-sample.bib: BibTeX file to store your bibliography.
  • pnas.csl and pnas-new.cls: Files containing information about the formatting of citations and bibliography adapted to journal policies.
  • frog.png: A .png file used to show you how to include figures in .Rmd document.
Screenshot of a macOS Finder list view showing seven items inside the template folder: frog.png, pnas-new.cls, pnas-sample.bib, pnas.csl, pnasresearcharticle.sty, Submission_PNAS.Rmd, and widetext.sty, with their sizes and file kinds.

Figure 7.3: Snapshot showing the suite of files associated to your submission and saved in your project folder.

  1. Start writing your article
  • Open Submission_PNAS.Rmd and update the YAML metadata section with information on authors, your abstract, summary and keywords (see Figure 7.4).
Close-up of the color-coded YAML header of an R Markdown file, listing the title field followed by an author block with two entries, each with affiliation codes, an address block mapping those codes to institutions, and a corresponding_author block with an email.

Figure 7.4: Update YAML metadata section with information on authors, your abstract, summary and keywords.

  • Write your manuscript by following the journal’s structure. You can take advantage of the R Markdown language in your manuscript (e.g. include R code chunks and outputs) and those will be converted by knitr and rticles packages during the compilation procedure.
  1. Compile your document and use both .pdf and .tex files to submit your article (see Figure 7.5). The output files will be saved in your project folder.
Screenshot of the RStudio editor toolbar with the Knit dropdown menu open over the Submission_PNAS.Rmd file, showing the highlighted Knit to pnas_article option above Knit with Parameters, Knit Directory, and Clear Knitr Cache.

Figure 7.5: Snapshot of the procedure to knit the document.

7.4.5.2 Exercise

To get familiar with this procedure, please practice by applying the above approach to different journal templates by favoring those where you might be submitting to.

Enjoy writing scientific publications in R Markdown!

8 Chapt. 7 – Organize and Import Data with R

8.1 Introduction

In this chapter, we will study protocols to import and gather data with R. As stated by Gandrud (2015) in the chapter 6 of his book, how you gather your data directly impacts how reproducible your research will be. In this context, it is your duty to try your best to document every step of your data gathering process. Reproduction will be easier if all of your data gathering steps are tied together by your source code, then independent researchers (and you) can more easily regather the data. Regathering data will be easiest if running your code allows you to get all the way back to the raw data files (the rawer the better). Of course, this may not always be possible. You may need to conduct interviews or compile information from paper based archives, for example. The best you can sometimes do is describe your data gathering process in detail. Nonetheless, R’s automated data gathering capabilities for internet-based information is extensive. Learning how to take full advantage of these capabilities greatly increases reproducibility and can save you considerable time and effort over the long run.

Gathering data can be done by either importing locally stored data sets (= files stored on your computer) or by importing data sets from the Internet. Usually, these data sets are saved in plain-text format (usually in comma-separated values format or csv) making importing them into R a fairly straightforward task (using the read.csv() function). However, if data sets are not saved in plain-text format, the users will have to start by converting them. In most cases, data sets will be saved in xls or xlsx formats and functions implemented in the readxl package (Wickham and Bryan, 2019) would be used (using the read_xlsx() function). If your data sets were created by other statistical programs such as SPSS, SAS or Stata, these could be imported into R using functions from the foreign package (R Core Team, 2020). Finally, data sets could be saved in compressed documents, which will have to be processed prior to importing the data into R.

Learning skills to import and gather data sets is especially important in the fields of Ecology, Evolution and Behavior since your research is highly likely to depend on large and complex data sets (see Figure 8.1). In addition, testing your hypotheses will rely on your ability to manage your data sets to test for complex interactions (e.g. does abiotic factors such as temperature drive selection processes in plants?; Figure 8.1).

Three overlapping circles in a Venn arrangement labeled Genome in red at upper left, Phenome in blue at upper right, and Environment in green at bottom center, with all three sharing a common central overlap plus three pairwise overlaps.

Figure 8.1: Example of datasets involved in Ecology, Evolution and Behavior and their intercations.

Here, we are providing methods and R functions that are applied to manage your projects and gather, convert and clean your data. Ultimately these tools will be applied to document and produce the raw data at the basis of your research.

8.2 Aim

This chapter aims to give you the procedures used to gather, import and organise data with R, so that the raw data behind a piece of research is something another person can obtain, verify and reuse.

It is written to be worked through on your own, at the console, in a project of your own. Every section is a short procedure with the code beside it.

8.3 Learning Outcomes

By the end of this chapter, you will be able to:

8.4 Class Outline

Almost all of this chapter is worked through on your own. It is the reference you implement your own project against, and it is where the earlier chapters point when they say that a repair belongs in Chapter 7: the pseudocode you wrote for somebody else’s broken project in Chapter 5 is implemented with the procedures below.

One block is done in class

Block Time Who
Checking data integrity with md5, the single-file example 5 min Instructor, with the class

Everything else, on your own

Block Time Who
Manage your work in an RStudio project 10 min Students, on their own
Import a csv from GitHub, including what a SHA-1 hash identifies 20 min Students, on their own
What md5 is, and where you will meet it 8 min Read before the class above
The md5 check over a whole folder, and the Challenge 12 min Students, on their own
Retrieve a csv from an earlier commit 10 min Students, on their own
Download and process a whole repository 20 min Students, on their own
Total 72 min

8.5 Resources

Everything you need in order to work through this chapter is in this section. The R packages and the GitHub repositories used below are listed here, so set them up before you start.

8.5.1 Literature and Web Resources

This chapter is mostly based on the following book and web resources:

The R packages and GitHub repositories used in this chapter are listed below. Make sure they are all set up before you start, because every section from Getting started! onwards assumes them.

8.5.2 Software Requirements

Most of the functions that we will be using in this chapter are base R functions installed (by default) in the utils package (R Core Team, 2019). However, the following R package (and its dependencies) has to be installed on your computer prior to starting this tutorial: repmis (Gandrud, 2016). In the event that you wanted to import xsl or xlsx files into R, you would also have to install the readxl package (Wickham and Bryan, 2019).

8.5.3 Data and Repositories

8.5.3.1 EEB603_Reproducible_Science repository

This repository is dedicated to this course and is used to demonstrate the procedure to import csv files into R from GitHub repositories. More specifically, we will be importing different versions of the file (Timetable_EEB603_topic_tasks.csv) at the origin of the Timetable to study procedure associated to file versioning in GitHub. It is located at this URL (Uniform Resource Locator):

8.5.3.2 Sagebrush_rooting_in_vitro_prop

This repository is associated to Barron et al. (2020) and is used to demonstrate procedure to download whole GitHub repositories. We will be downloading the whole repository on your local computer and then extracting all csv files in the 01_Raw_Data/ folder and saving them on your local computer using R (see Figure 8.2 for more details on file content).

Screenshot of a GitHub folder listing for the 01_Raw_Data directory of the Sagebrush_rooting_in_vitro_prop repository, with a header row showing the last commit and a table of eight files including five block CSVs, Phenotypes_sagebrush_in_vitro.csv, and Survival_height_clones.csv, each with its commit message and age.

Figure 8.2: Snapshot showing files in the 01_Raw_Data folder that we will be targeting in the GitHub repository.

This approach is aimed at demonstrating how you could access raw data deposited on GitHub. The repository is located at this URL:

8.5.4 Overview of Methods Applied in This Chapter

In this chapter, we will be focusing on learning procedures to import data from the internet focusing on GitHub. More precisely, we will be learning procedures to:

  1. Import comma-separated values format (csv) stored in GitHub repositories.
  2. Verify that the data you imported is the data you were given, using md5 checksums.
  3. Download and process whole GitHub repositories saved in compressed zip format.

8.5.5 List of R Functions Covered in This Chapter

The list of major R functions covered in this chapter are provided in the Table below.

8.6 Managing projects in RStudio

RStudio projects (.Rproj) allow users to manage their project, more specifically by dividing their work into multiple contexts, each with their own working directory, workspace, history, and source documents.

8.6.1 Creating projects

RStudio projects are associated with R working directories. You can create an RStudio project:

  1. In a brand new directory.
  2. In an existing directory where you already have R code and data.
  3. By cloning a version control (Git or Subversion) repository.

We will be covering the last option during Chapter 11.

To create a new project in RStudio, do File > New Project... and then a window will pop-up allowing you to select among the 3 options (see Figure 8.3).

Screenshot of the RStudio New Project Wizard dialog titled Create Project, offering three stacked options with icons and descriptions: New Directory to start in a brand new working directory, Existing Directory to associate a project with an existing directory, and Version Control to check out a project from a repository.

Figure 8.3: Window allowing you to create a New RStudio project. See text for more details.

When a new project is created RStudio:

  1. Creates a project file (with an .Rproj extension) within the project directory. This file contains various project options and can also be used as a shortcut for opening the project directly from the filesystem.
  2. Creates a hidden directory (named .Rproj.user) where project-specific temporary files (e.g. auto-saved source documents, window-state, etc.) are stored. This directory is also automatically added to .Rbuildignore, .gitignore, etc. if required.
  3. Loads the project into RStudio and display its name in the Projects toolbar (which is located on the far right side of the main toolbar).

8.6.2 Working with projects

To open a project, go to your project directory and double-click on the project file (*.Rproj). When your project opens within RStudio, the following actions will be taken:

  • A new R session (process) is started.
  • The current working directory is set to the project directory.
  • Previously edited source documents are restored into editor tabs.
  • If you saved your workspace into a .RData file (see below), it will be loaded into your environment and allowing you to purse your work.

When you are within a project and choose to either Quit, close the project, or open another project the following actions will be taken:

  • .RData and/or .Rhistory are written to the project directory (if current options indicate they should be).
  • The list of open source documents is saved (so it can be restored next time the project is opened).
  • The R session is terminated.

8.6.3 Additional information

Additional information on RStudio projects can be found here:

https://support.rstudio.com/hc/en-us/articles/200526207-Using-Projects

8.7 Getting started!

Before starting coding, please do the following:

  1. Open RStudio and create a New Project in a New Directory entitled EEB603_Chapter_07.
  2. Open the new project and create and save a new R script entitled 01_Data_gathering.R.

8.8 Importing csv files from GitHub into R

Before delving into this subject, there are several topics that we need to cover.

8.8.1 The SHA-1 hash accession number

With the growing popularity of GitHub, several authors are depositing their data sets on GitHub and you might would like to access those for your research. Since Git and GitHub are supporting version control, it is important to report the exact version of the file or data set that you have downloaded. To support such feature, each version of a file/data set is associated to a unique encrypted SHA-1 (Secure Hash Algorithm 1) hash accession number. This means that if the file changes (because the owner of the repository updated it), its SHA-1 hash accession number will change. Such feature allows users to referring to the exact file/data set used in their analyses therefore supporting reproducibility.

8.8.2 Checking Data Integrity with md5

A download can arrive incomplete, and an open file can be saved back changed, and neither of those announces itself. The file is still there, it still opens, and read.csv() still returns a data frame. What you need is a short piece of evidence, recorded while the data was in the state you intended, that can be re-checked at any point afterwards.

An md5 checksum is that evidence: a 32-character hash computed from the bytes of a file. Change one character of the file and the hash changes completely. Two uses follow, and they are why this section is here:

  1. It tells you a download arrived intact. Compare the hash of what landed on your disk with the hash the provider published.
  2. It tells you your raw data has not been touched since. Chapter 5 asked you to treat 01_Raw_Data/ as written once and never again. A checksum is how that claim gets checked rather than asserted.

The convention is a companion file named MD5 sitting beside the data, with one line per file: the hash, a space, an asterisk, then the file name. R’s own packages ship exactly this file, which is why the function that verifies it is already installed on your machine.

💡 md5 and SHA-1 answer different questions

The SHA-1 hash in the previous section is GitHub’s: it identifies a version of a file in a repository, and Git assigns it. The md5 hash here is yours, and it identifies the bytes sitting on your own disk. The first lets you cite the version you used; the second lets you show that what you analysed is that version.

8.8.2.1 Where md5 Comes From, and Where You Will Meet It

md5 is the fifth of Ronald Rivest’s Message Digest algorithms, designed in 1991 to replace MD4 and published in April 1992 as RFC 1321 from MIT and RSA Data Security. The specification’s own word for the output is a fingerprint: input of any length in, 128 bits out, which is the 32 hexadecimal characters you have been looking at. Three properties make it useful here:

  • Deterministic. The same bytes give the same digest on any machine, in any language, in any year. That is why a hash a sequencing centre computed in 2019 still means something on your laptop today.
  • Fixed length. A 2 GB fastq and a three-line csv both reduce to 32 characters, so the record costs nothing to store, publish or paste into a methods section.
  • Avalanche. Change one bit and about half the output bits change. There is no such thing as two digests being nearly the same: they either match or they do not.

💡 md5 is broken, and it is still the right tool for this job

RFC 1321 was written for digital signatures, and for that purpose md5 is dead. Dobbertin found a collision in its compression function in 1996, practical attacks followed in the mid-2000s, and RFC 6151 (2011) states that md5 is no longer acceptable where collision resistance is required.

A collision is two files deliberately built to share a digest. It takes an adversary who wants to deceive you, and it is not what goes wrong with research data. What goes wrong is a transfer that stopped at 94%, a disk that flipped a bit, a file opened in Excel and saved back. None of those produces a collision, so md5 catches every one of them, which is why the archives below still ask for it.

The rule: use md5 to detect accident, never to prove authenticity. If you need the second, for signing a release or storing credentials, use SHA-256 or better and not md5.

Where you will meet it. A checksum is not a classroom exercise; it is part of the ordinary paperwork of sequence data.

Where What it is for
Sequencing facilities A fastq delivery ships with a .md5 file. Checking it after the transfer is how you discover that a 40 GB download truncated, before you spend a week aligning it
ENA submissions An md5 per file is required in the manifest (forward_file_md5, reverse_file_md5) and must be lower case, so the archive can confirm your upload completed without error
NCBI genomes FTP Every assembly directory carries an md5checksums.txt covering its data files. The current human reference, GRCh38.p14, lists 1,388 of them
BAM and CRAM The @SQ header’s M5 tag is the md5 of the reference sequence itself, uppercased with gaps and spaces removed (SAM specification). Optional in SAM and BAM, mandatory in CRAM, which uses it to locate the right reference
R packages Every CRAN package ships an MD5 file, and that is what checkMD5sums() was written to verify

The BAM and CRAM row is worth a second read, because it is this section’s idea one level up. Rather than trusting that two files named hg38.fa hold the same genome, the format records a digest of the sequence and compares that. A file name is a claim; a checksum is evidence. That distinction is the whole of Chapter 5 PART A in one line.

8.8.2.2 Step-by-Step Procedure (~12 minutes)

Both functions live in the tools package, which ships with R, so there is nothing to install. It is not attached by default, which is why every call below is written tools::. Work in the project you created in Getting started!.

💡 How this one runs

Read Where md5 Comes From first. My five minutes in class are the mechanism, not an introduction to hashing.

Example 1, on a single file, is done together in class. It is the whole mechanism in three lines, and it is short.

Example 2 and the Challenge are yours to work through afterwards. Example 2 repeats Example 1 over a whole folder, which is the form you will actually use, and the Challenge breaks the check on purpose so that you have seen it fail before it matters.

First, put a raw data file in place. Any csv will do; this one is made from scratch so that the exercise does not depend on a download.

### ~~~ Create the raw data folder and a small csv standing
### in for your own data ~~~
dir.create("01_Raw_Data", showWarnings = FALSE)
d <- data.frame(plant_id = c("A1", "A2", "A3"), mass_g = c(10.2,
    11.5, 9.8))
write.csv(d, "01_Raw_Data/species_data.csv", row.names = FALSE)
8.8.2.2.1 Example 1: one file (~5 minutes, in class)

1. Hash the file. tools::md5sum() returns a named vector: the value is the hash and the name is the path you asked about.

hashes <- tools::md5sum("01_Raw_Data/species_data.csv")
hashes
      01_Raw_Data/species_data.csv
"2a4868210cf93dabb09d2777993380a4"

Your hash need not be the one printed here, because a text file’s bytes are not identical on every operating system. That is not a problem: a checksum is only ever compared with one computed from the same file, never with somebody else’s.

2. Write the companion MD5 file. One line, in the order the checker expects: the hash, a space, an asterisk, then the file name. basename() drops the folder, so the manifest describes the file relative to the folder it sits in and stays valid if the folder is copied elsewhere.

writeLines(paste0(hashes, " *", basename(names(hashes))), "01_Raw_Data/MD5")
2a4868210cf93dabb09d2777993380a4 *species_data.csv

3. Verify it. tools::checkMD5sums() reads the MD5 file in the folder you give it, re-hashes what is there, and compares the two.

tools::checkMD5sums(dir = normalizePath("01_Raw_Data"))
[1] TRUE

TRUE is the only answer that means all is well. The function returns FALSE when something is missing or changed, and NA when it cannot find an MD5 file at all, so in a script write isTRUE(tools::checkMD5sums(...)): a manifest that has gone missing should fail the check, not pass it.

💡 Pass dir an absolute path, or “.”

checkMD5sums() changes directory into dir and then lists dir again from inside it, so a relative path other than “.” finds nothing at all and the function reports every file as missing: 

tools::checkMD5sums(dir = "01_Raw_Data")
file 'species_data.csv' is missing
[1] FALSE

Nothing is actually missing. normalizePath(“01_Raw_Data”) makes the path absolute and the check then works, and dir = “.” works when the MD5 file sits in your working directory. A check that reports failure for the wrong reason is worse than no check, because you learn to ignore it.

8.8.2.2.2 Example 2: every file in the folder (~7 minutes, on your own)

Raw data is rarely one file. Work through this one yourself, after class. Nothing above changes except what you hand to md5sum(), because it is vectorised: give it many paths and you get many hashes back, still named.

1. Add a second data file, so that the folder holds two.

temp <- data.frame(site = c("S1", "S2"), temp_c = c(22.4, 19.8))
write.csv(temp, "01_Raw_Data/temperature.csv", row.names = FALSE)

2. Run the check again before you change anything else. Think about what it should say, then run it.

tools::checkMD5sums(dir = normalizePath("01_Raw_Data"))
[1] TRUE

💡 It said TRUE, and there is a file it has never heard of

checkMD5sums() only checks the files the manifest lists. It will tell you that a listed file has changed or gone missing, and it is completely blind to a file that was added. The manifest you wrote in Example 1 names one file, that file is intact, so the answer is honestly TRUE.

This is worth knowing before you rely on it: the check is only as complete as the list. Whenever the contents of the folder change on purpose, the manifest has to be rewritten, which is exactly what the next step does.

3. Hash everything at once. list.files() builds the paths and md5sum() takes all of them in one call.

### ~~~ Every csv in the raw data folder, in one call ~~~
files <- list.files("01_Raw_Data", pattern = "\\.csv$", full.names = TRUE)
files
hashes <- tools::md5sum(files)
hashes
[1] "01_Raw_Data/species_data.csv" "01_Raw_Data/temperature.csv"

      01_Raw_Data/species_data.csv        01_Raw_Data/temperature.csv
"2a4868210cf93dabb09d2777993380a4" "a133cc888631b7c27badfa21ad5997ee"

4. Rewrite the manifest and verify. The line that writes the file is the same line as in Example 1: paste0() is vectorised too, so it produces one line per file without a loop.

writeLines(paste0(hashes, " *", basename(names(hashes))), "01_Raw_Data/MD5")
tools::checkMD5sums(dir = normalizePath("01_Raw_Data"))
2a4868210cf93dabb09d2777993380a4 *species_data.csv
a133cc888631b7c27badfa21ad5997ee *temperature.csv

[1] TRUE

Now TRUE means both files, and that is the state you want your raw data folder to be in before you analyse anything out of it.

8.8.2.3 Challenge (~5 minutes, on your own)

💡 Break it on purpose

  • Open 01_Raw_Data/species_data.csv, change one mass value, and save it. A spreadsheet or a text editor is fine, and so is doing it from R.
  • Run tools::checkMD5sums() again. What does it return, and what does it print?
  • Now delete the file and run the check a third time. Is the message the same one?
  • Then answer this: you have changed the data and the check has told you so. Which of the two files do you now fix, and what does that tell you about when the MD5 file should be written?
✅ Solution

An edited file comes back FALSE, with the file named:

file 'species_data.csv' has the wrong MD5 checksum
[1] FALSE

A deleted file comes back FALSE too, but the message is different, and the difference matters: is missing tells you the file is gone, while has the wrong MD5 checksum tells you the file is there and is not what it was.

file 'species_data.csv' is missing
[1] FALSE

Which file do you fix? Neither, until you know what happened. The checksum has not told you that the data is wrong; it has told you that the data and its record of itself disagree. Either the file changed when it should not have, in which case you restore it, or the change was intended, in which case the MD5 file is now out of date and you regenerate it deliberately.

So the MD5 file is written once, when the raw data arrives, and regenerated only on purpose. Regenerating it every time you run the analysis makes it agree with whatever is on disk, which means it can never disagree, which means it is no longer evidence of anything. That is the whole mechanism, and it is the same argument as treating raw data as read-only: a record that updates itself to match reality cannot detect reality changing.

One more thing worth noticing. Re-saving a csv from Excel changes its bytes even when you change no value, so the hash changes. That is not a false alarm. It is the check doing its job, and it is exactly the defect Pair 4 diagnosed in Chapter 5.

From here on, do this with real data. When you download the csv in the next section, and the whole repository further down, write an MD5 file beside what you receive. It costs two lines and it is the only way to be able to say, months later, that the file you analysed is the file you were given.

8.8.3 How to retrieve the URL of a GitHub csv file?

Before being able to import a csv file deposited in GitHub into R you have to find it’s raw URL. In this section, we will demonstrate how to obtain this information by using the Timetable_EEB603_topic_tasks.csv file located on the course’s GitHub repository.

To retrieve the raw URL associated to Timetable_EEB603_topic_tasks.csv do the following:

  1. Navigate to the file location on the GitHub repository by clicking here (see Figure 8.4).
    Screenshot of the GitHub file view for Timetable_EEB603_topic_tasks.csv in the Data folder of the EEB603_Reproducible_Science repository, with the breadcrumb path and Raw and Blame buttons above a rendered two-column table of Topic and Task Deadline entries.

    Figure 8.4: Location of Timetable_EEB603_topic_tasks.csv on the EEB603_Reproducible_Science GitHub repository.

  2. Click on the Raw button on the right just above the file preview (Figure 8.4). This action should open a new window showing you the raw csv file (see Figure 8.5).
    Screenshot of a browser window at raw.githubusercontent.com displaying the plain-text CSV contents, starting with the header line Topic,Task Deadline and followed by comma-separated rows pairing course topics with their deadlines.

    Figure 8.5: Raw csv file associated to Timetable_EEB603_topic_tasks.csv. See text for more details.

  3. The raw URL for this csv file can be retrieved by copying the URL address (see Figure 8.5). In this case, the URL is as follows: https://raw.githubusercontent.com/svenbuerki/EEB603_Reproducible_Science/master/Data/Timetable_EEB603_topic_tasks.csv
  4. The URL can be shorten for free by using Bitly at https://bitly.com. In this case, the short URL pointing to our csv file is https://bit.ly/3BDxECl. We will be using this URL in the example below.

8.8.4 Importing GitHub csv into R

Now that we have retrieved and shortened the raw GitHub URL pointing to our target csv file, we can use the source_data() function implemented in the R package repmis (Gandrud, 2016) to download the file. The object return by source_data() is a data.frame and can therefore be easily manipulated and saved on your local computer (using e.g. write.csv()). Retrieving a csv data set from a GitHub repository can be done as follows:

### ~~~ Load package ~~~
library(repmis)
### ~~~ Store raw short URL into object ~~~
urlcsv <- "https://bit.ly/3BDxECl"
### ~~~ Download/Import csv into R ~~~
csvTimeTable <- repmis::source_data(url = urlcsv)
## Downloading data from: https://bit.ly/3BDxECl
## SHA-1 hash of the downloaded data file is:
## 4f5ae70a7161ab66b2b392832883bf2a7957e10b
### ~~~ Check class ~~~ Class should be `data.frame`
class(csvTimeTable)
## [1] "data.frame"
### ~~~ Print csv ~~~
print(csvTimeTable)
##                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Topic
## 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            [Syllabus](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html)
## 2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [Chapt. 1 - part A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#WhatRepSci)
## 3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [Chapt. 1 - part A: start at section 2.4.3 (Case Studies and Challenges)](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#CaseStudies)
## 4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Complete [Chapt. 1 - part A: start at section 2.4.5 (Wrap-up and Future Directions)](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#WrapUp) and [Chapt. 1 - part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#WhatFacBreakRep)
## 5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Complete [Chapt. 1 - part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#WhatFacBreakRep) and start [Chap. 2 - part A: Learning the basics](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA)
## 6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         Complete [Chapt. 1 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#commitcharter) (starting at section 2.5.5.7 Revise and Commit) and start [Chap. 2 - Part A: Learning the Basics](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA)
## 7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             Complete [Chap. 2 - Part A: Learning the Basics](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA)
## 8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          [Chap. 2 - Part B: Setting Your R Markdown Document](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partB) (up to the end of [Install R Dependencies and Load Packages](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#loadRreq))
## 9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Complete [Chap. 2 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partB) (starting at [Generate a Citation File of R Packages Used](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#citfileRpkg)) and start [Chap. 2 - Part C: Tables, Figures and References](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC)
## 10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        [Individual project workshop](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Report): develop your project and proposal with feedback from me
## 11                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Review [Chap. 2 - Part C: Tables, Figures and References](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC), then [Chap. 2 - Part D: Advanced R and R Markdown Settings](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partD)
## 12                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    [Insert a Figure from an Image File](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#figurefromfile): a 20-minute challenge filling a gap in Part C, then [Chap. 2 - Part E: User-Defined Functions in R](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE): what a function is, and your first UDF
## 13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Complete [Chap. 2 - Part E: User-Defined Functions in R](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE), starting at [Create, Name, and Access Elements of a List in R](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#Rlists)
## 14                                       Complete [Chap. 2 - Part E](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE) **in full**: pairs present their code for [UDFs that Return Multiple Values](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#UDFmultiple) at the whiteboard, then [Logical Operators](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#logicalop), [Defensive Programming](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#defensiveprog) and [Use Your UDFs in an R Markdown Document](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#UDFinRmd). We then start [Chap. 3: A Roadmap for Reproducible Science](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap3): the [steps of the scientific process](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#scientificprocess) and the [threats to it](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#threatstoprocess)
## 15                                                                                                                                                                                                                                                                                                                  [Chap. 3](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap3): a 15-minute synthesis of the [steps of the scientific process](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#scientificprocess) and the [threats to it](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#threatstoprocess) from the slides, then the group activities [Do Your Standards Hold?](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#charterthreats) and [Score Your Charter Against TOP](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chartertop), with the [TOP Guidelines](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#TOP) in between
## 16                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              **Complete [Chap. 3](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap3)** with the [presentations of both group activities](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#reportbacktop) **in the first 10 minutes**, **then [Chap. 4: Open Science and CARE Principles](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap4)**
## 17                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              **Complete [Chap. 4](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap4)** with the [FAIR pairs' report](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#FAIRactivity) and the [CARE Principles activity](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#CAREactivity), **then [Chap. 5 - PART A: Data Management](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partDM)**
## 18                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      **Complete [Chap. 5 - PART A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partDM)** with [Steps 2, 3 and 4](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#importcsv): open a csv, build the [Defect_log and the metadata](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#metadataskeleton), and start diagnosing the area your pair owns
## 19                                                                                                                                                                                                                                                                                                                                                                           **&#128187; On Zoom.** **[Chap. 5 - PART B: Reproducible Code](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#repcode)**, first of two classes: the [One Pass Over the Workflow](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codespine), then [Step 1: Try to Run It](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep1) and [Step 2: Diagnose the Area Your Pair Owns](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep2). **The [reporting round](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codereporting), which also closes PART A, moves to the next class**
## 20 **&#128187; On Zoom.** **Finish the [Chap. 5 - PART B group activity](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codereporting)**: [Step 3: report](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep3) your data and code defects together, [Step 4: write the clauses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep4), [Step 5: merge into your charter](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep5). **Then we begin [Chap. 6: Getting Published and Peer Review](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap6outline)** with [Open Access](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#openaccess), [Creative Commons Licenses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#CClicenses) and [Software Licenses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#softwarelicenses), the licences on the article, the data and the code
## 21                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 **First 20 minutes: [Chap. 6 - PART A: Getting published](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#getpub)**, the four steps from selecting a journal to acceptance. **Then a work session on your bioinformatics tutorials**: bring your tutorial in progress; I am there to give feedback as you build it
## 22                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       **Work session: individual projects.** I am away this day, so use the class time to build your report. Bring anything you want me to look at to the following class
## 23                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Bioinfo. tutorial - Chap. 9: **[get the material here](https://drive.google.com/drive/folders/12SJOh-qvsr3w1unBgM1vTd39_IZzHOal?usp=drive_link)** and work through it before class
## 24                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Bioinfo. tutorial - Chap. 9: **[get the material here](https://drive.google.com/drive/folders/12SJOh-qvsr3w1unBgM1vTd39_IZzHOal?usp=drive_link)** and work through it before class
## 25                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 10: **[get the material here](https://drive.google.com/drive/folders/1TUM1XNh_cRQJjqxTmJ3Knbp7pWsPQS0v?usp=drive_link)** and work through it before class
## 26                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 10: **[get the material here](https://drive.google.com/drive/folders/1TUM1XNh_cRQJjqxTmJ3Knbp7pWsPQS0v?usp=drive_link)** and work through it before class
## 27                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 12: **[get the material here](https://drive.google.com/drive/folders/1S2LpQ6CMu6lLbq-xdEyYdhhdmW4Ox9l2?usp=drive_link)** and work through it before class
## 28                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 12: **[get the material here](https://drive.google.com/drive/folders/1S2LpQ6CMu6lLbq-xdEyYdhhdmW4Ox9l2?usp=drive_link)** and work through it before class
## 29                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 11: **[get the material here](https://drive.google.com/drive/folders/19o-oLicEFxHFYxFL65ue1OhaveVL0Cl7?usp=drive_link)** and work through it before class
## 30                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Bioinfo. tutorial - Chap. 11: **[get the material here](https://drive.google.com/drive/folders/19o-oLicEFxHFYxFL65ue1OhaveVL0Cl7?usp=drive_link)** and work through it before class
## 31                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Oral presentations of individual projects
## 32                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Oral presentations of individual projects
##                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Homework
## 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
## 2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  <ul><li>Read the syllabus carefully</li><li>Note any questions you have, and ask them in class</li></ul>
## 3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               <ul><li>Read your assigned publication for Chapter 1 - part A (on [Google Drive](https://drive.google.com/drive/folders/1AeoPIzzs4mEUMIhUiACT7TYtUHS9U0Nu?usp=drive_link))</li><li>Identify its reproducibility challenges, ready to discuss them with your group</li></ul>
## 4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  <ul><li>Read Baker (2016) (on [Google Drive](https://drive.google.com/drive/folders/13aOxsvVgyxA3sm8bf1RBXLvrR5nCHVhp?usp=drive_link))</li><li>Read the Summary (pages 5-19) of the <i>Reproducibility and Replicability in Science</i> report, from The National Academies Press (on [Google Drive](https://drive.google.com/drive/folders/13aOxsvVgyxA3sm8bf1RBXLvrR5nCHVhp?usp=drive_link))</li></ul>
## 5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     <ul><li>[Sign up for a bioinformatics tutorial](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#GoogleBioTut)</li><li>Finish your group’s draft [reproducibility charter](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#draftcharter) in the shared Google Doc</li><li>[Install software](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#InstallSoftware)</li></ul>
## 6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              <ul><li>Complete section [2.5.5.7 Revise and Commit](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#commitcharter) of your group’s charter</li></ul>
## 7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <ul><li>Complete section [3.2.4 Your First R Markdown Document](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#firstMDhow) of Chapter 2 - Part A</li></ul>
## 8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          <ul><li>Complete [Chap. 2 - Part A: Learning the Basics](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA)</li></ul>
## 9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  <ul><li>Complete [Chap. 2 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partB) up to the end of [Install R Dependencies and Load Packages](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#loadRreq)</li></ul>
## 10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   <ul><li>Complete [Chap. 2 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partB) to the end, through [Generate Appendix with R Package Versions and Operating System](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#appendixRverOS)</li><li>Check your document against the [finished template](https://github.com/svenbuerki/EEB603_Reproducible_Science/blob/master/Exercises/Chapter_2/Chapter_2_partB_template.Rmd)</li><li>Read the [Project Proposal assignment](https://svenbuerki.github.io/EEB603_Reproducible_Science/Assignments.html#project-proposal-50-points) and its [rubric](https://svenbuerki.github.io/EEB603_Reproducible_Science/Evaluation_Forms/Project_Proposal_Evaluation_Form.pdf) before class</li><li>Bring your scientific question, your candidate dataset (yours or a publication), and anything you have already drafted or coded</li></ul>
## 11                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <ul><li>Complete [Chap. 2 - Part C: Tables, Figures and References](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC)</li><li>If your document does not knit, download the [finished Part C document](https://github.com/svenbuerki/EEB603_Reproducible_Science/blob/master/Exercises/Chapter_2/Chapter_2_partC_template.Rmd) and bring it, so you can start Part D on time</li><li>Start drafting your [individual project proposal](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Proposal)</li></ul>
## 12                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <ul><li>Finish [Chap. 2 - Part D](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partD): steps 6 to 9 of [Implement Global Options for Code Chunks](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#setupcode), the plot chunk and its cross-reference</li><li>Then complete the [Challenge](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#globalchallenge) at the end of Part D</li><li>Continue drafting your [individual project proposal](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Proposal)</li></ul>
## 13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          <ul><li>Finish [UDFs That Return a Single Value](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#UDFsingle): implement `square_number.R` and apply it, then check your work against that section's Checkpoint</li><li>**Your [individual project proposal](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Proposal) is due by 5 PM on Thursday, 8 October.** **It must be an R Markdown source that knits** --- not a PDF or a Word file --- so leave time to check that it does. Bring any questions to Tuesday's class: it is your last chance to ask before you submit</li></ul>
## 14                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               <ul><li>**Make sure you have completed [Develop, Implement, and Apply UDFs that Return Multiple Values](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#UDFmultiple)**: your pair presents its plan and its code at the whiteboard at the start of class, so the function needs to be written and tested</li><li>**Finalise and submit your [individual project proposal](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Proposal) by 5 PM.** **Hand in a folder named with your first and last name** (for example `Jane_Doe`) containing the `.Rmd` source and everything needed to knit it: the `.bib`, the `.csl`, any data and any images. See [How to Submit It](https://svenbuerki.github.io/EEB603_Reproducible_Science/Assignments.html#proposalsubmit). Before you submit, check that it knits without error, contains at least one code chunk that runs, and manages its citations through a bibliography file. **A PDF or Word file cannot earn the marks reserved for a working reproducible document**</li></ul>
## 15                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       <ul><li>Complete [Use Your UDFs in an R Markdown Document](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#UDFinRmd), the last section of [Chap. 2 - Part E](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE)</li><li>Read [Discuss Solutions to Mitigate Threats](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#soltothreats). This is not background reading: the first group activity asks you to write new clauses for your charter, and a clause naming an established practice is far stronger than one you invent</li><li>Read the [TOP Guidelines](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#TOP) section before class</li></ul>
## 16 <ul><li>**Finish updating your charter** from the [Do Your Standards Hold?](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#charterthreats) activity: every threat tested, the clauses written, and the remaining gaps recorded as dated to-dos</li><li>**Complete the [journal comparison](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#comparejournal)**: each member takes a different journal they might realistically publish in and finds its TOP policy, noting one place it is stricter than your charter and one place your charter is stricter than it</li><li>**Prepare a five-minute group presentation for the start of class**, covering [both Chapter 3 activities](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#reportbacktop): the threat your charter handled best, the gap that surprised you most and the clause you wrote, your highest and lowest TOP standards, and what the journal comparison showed</li><li>**Pre-read the two frameworks the activities use**: [FAIR Data](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#FAIR), [The CARE Principles for Indigenous Data Governance](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#CAREPrinc) and [Data Sovereignty](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#datasov). The three activities done in class apply these rather than introduce them, so a pair arriving without them spends its fifteen minutes looking up definitions instead of thinking</li></ul>
## 17                                                                                                                                                                                                                                                                                                                     <ul><li>**Pre-read the seven stages of the data life cycle**: [Planning](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#planningDM), [Creating](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#creatingdata), [Processing](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#processingdata), [Documenting](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#documentingdata), [Preserving](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#preservingdata), [Sharing](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#sharingdata) and [Reusing](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#reusingdata) data. My pass over them in class is a synthesis, not your first contact with them</li><li>**Complete the [Before Class Activity](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#beforeclass5A)**: create a `Chapter_5` folder in your `Chapters` folder, beside `Chapter_2`, start a script named `Chapter_5_data_management.R`, and use it to download and unzip `Messy_Project.zip`. **Do not tidy the project up**: its state is the exercise. Bring your laptop</li></ul>
## 18                                                                                                                                                                                                                                                                         <ul><li>**Complete [What to Set Up on Your Computer Before Class](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#setup5B)**: start a second script named `Chapter_5_reproducible_code.R` in your `Chapter_5` folder and run [Now Diagram the Project You Inherited](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#messystructure) in it. **Install the `data.tree` and `DiagrammeR` packages before class** ([installation code here](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#RpkgRepCode)): installing them during the session costs everybody ten minutes. **Bring your laptop**</li><li>**Optional: revise and resubmit your [individual project proposal](https://svenbuerki.github.io/EEB603_Reproducible_Science/Assignments.html#proposalrevise) by 5 PM** to earn up to the full 50 points. Address the comments on your evaluation, then download the [rubric](https://svenbuerki.github.io/EEB603_Reproducible_Science/Evaluation_Forms/Project_Proposal_Evaluation_Form.pdf), fill in how you addressed each section, award yourself a score for each, and upload it with your revised folder on [Google Drive](https://drive.google.com/drive/folders/1djFlKe7Lz13HnOSV-ThXtR18_3sS8Lm2?usp=drive_link). Entirely optional</li></ul>
## 19                                                                        <ul><li>**&#128187; Join on Zoom**, [invite and meeting link here](https://docs.google.com/document/d/1c1Dc661xqO-sIy4qZ39nM5_LCE7mobgtF_KS6aJQ1YU/edit?usp=sharing). **Test your audio before 9:00** so the session starts on time</li><li>**Finish [Step 4: diagnose the area your pair owns](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#inheritstep2).** Your pair's rows must be in `Defect_log.csv`, with the `fix_pseudocode` column filled, and the metadata columns you can fill completed in `README_copy_of_data_1.csv`. **Knit the `.Rmd` before class**</li><li>**Pre-read the four sections listed in [What to Read Before Class](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#preread5B)**: [Organize Your Project for Reproducibility](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#orgprjrep), [Apply Basic Programming Standards](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#basiccodestdr) (the longest and most important), [Code Portability](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codeport) and [Document and Manage Software Dependencies](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#docsoftdep). [Software Licenses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#softwarelicenses) is half a page, read it too</li><li>Keep working on your bioinformatics tutorial and individual project</li></ul>
## 20                       <ul><li>**&#128187; Join on Zoom**, [invite and meeting link here](https://docs.google.com/document/d/1c1Dc661xqO-sIy4qZ39nM5_LCE7mobgtF_KS6aJQ1YU/edit?usp=sharing). **Test your audio before 9:00** so the session starts on time</li><li>**Finish [Step 2: diagnose the area your pair owns](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep2).** Your pair's code defects must be rows in `Defect_log.csv`, with the `fix_pseudocode` column filled, under a `# Code defects` heading in `Defect_log.Rmd`. **Knit it before class**, because [Step 3](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#codestep3) is read straight off it, and have your group's [charter](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#FactorsActivity) and the [`Clauses` sheet](https://docs.google.com/spreadsheets/d/1nK4LA6lnjxn5nrbpsfq9hiYKQf7ij8vywdtwq3qbAuY/edit?usp=sharing) open</li><li>**Re-read [Open Access](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#openaccess) and [Creative Commons Licenses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#CClicenses)** from Chapter 4 and **[Software Licenses](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#softwarelicenses)** from Chapter 5. I go over the three of them together, because an article, its data and its code are licensed under three different instruments</li><li>Keep working on your bioinformatics tutorial and individual project</li></ul>
## 21                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     <ul><li>**Bring the work itself, not questions about it.** The session is for building, with me available as you go</li><li>**Read [PART A: Getting published](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#getpub)** before class, since my twenty minutes on it is a synthesis rather than an introduction, and **bring a journal you might realistically submit to**</li><li>**Read [PART B: Writing papers in R Markdown](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#writpub)**, which is not covered in class, with the [slides](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#teachmat6B)</li><li>Keep working on your bioinformatics tutorial and individual project</li></ul>
## 22                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 <ul><li>**Use the session to write.** The [final report](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Report) is due 3 December, and this is protected time for it</li><li>Keep working on your bioinformatics tutorial</li></ul>
## 23                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 24                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 25                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 26                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 27                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 28                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Students work on bioinformatics tutorials and their individual projects
## 29                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Revise your report using my feedback; prepare your presentation
## 30                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               Finalise your presentation
## 31                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
## 32                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
##                                                                                                                                                                                                                                                                                                                                      Deadline
## 1                                                                                                                                                                                                                                                                                                                                            
## 2                                                                                                                                                                                                                                                                                                                                            
## 3                                                                                                                                                                                                                                                                                                                                            
## 4                                                                                                                                                                                                                                                                                                                                            
## 5                                                                                                                                                                                                                                                                                                                                            
## 6                                                                                                                                                                                                                                                                                                                                            
## 7                                                                                                                                                                                                                                                                                                                                            
## 8                                                                                                                                                                                                                                                                                                                                            
## 9                                                                                                                                                                                                                                                                                                                                            
## 10                                                                                                                                                                                                                                                                                                                                           
## 11                                                                                                                                                                                                                                                                                                                                           
## 12                                                                                                                                                                                                                                                                                                                                           
## 13                                                                                                                                                                                                                                                                                                                                           
## 14                                                                                                                                                                                                                                                                                                    **Turn in individual project proposal**
## 15                                                                                                                                                                                                                                                                                                                                           
## 16                                                                                                                                                                                                                                                                                                                                           
## 17                                                                                                                                                                                                                                                                                                                                           
## 18                                                                                                                                                                                                                                                                                 **Turn in revised individual project proposal** (optional)
## 19                                                                                                                                                                                                                                                                                                                                           
## 20                                                                                                                                                                                                                                                                                                                                           
## 21                                                                                                                                                                                                                                                                                                               Turn in tutorial for Chap. 9
## 22                                                                                                                                                                        Upload tutorial of Chap. 9 on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Bioinformatics_tutorials`
## 23                                                                                                                                                                                                                                                                                                              Turn in tutorial for Chap. 10
## 24                                                                                                                                                                       Upload tutorial of Chap. 10 on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Bioinformatics_tutorials`
## 25                                                                                                                                                                                                                                                                                                              Turn in tutorial for Chap. 12
## 26                                                                                                                                                                       Upload tutorial of Chap. 12 on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Bioinformatics_tutorials`
## 27                                                                                                                                                                                                                                                                                                              Turn in tutorial for Chap. 11
## 28 **Turn in individual projects** (on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Individual_reports`); Upload tutorial of Chap. 11 on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Bioinformatics_tutorials`
## 29                                                                                                                                                                                                                                                                                                                                           
## 30                                                                                                                                                                  Upload slides of individual projects (on [Google Drive](https://drive.google.com/drive/folders/1eZyjGQ478vaKiGD01hcryyNtdGp9t1J7?usp=drive_link) -- `Individual_reports`)
## 31                                                                                                                                                                                                                                                                                                                                           
## 32                                                                                                                                                                                                                                                                                                                                           
##                                                                                                                                                                                                                 URL
## 1                                                                                                                                   [Syllabus](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html)
## 2                             [Chapter 1](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap1) & [Chapter 2](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap2)
## 3                                                                                                                            [Part A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA)
## 4  [Part A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partA) & [Part B: Set your R Markdown environment](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partB)
## 5                                                                                                                            [Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC)
## 6                                                                                                                            [Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC)
## 7                                   [Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partC) & [Part C](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partD)
## 8                                                                                                                            [Part D](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE)
## 9                                                                                                                            [Part D](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partE)
## 10                                                                                                                                                                                                                 
## 11                                                                             [Publications and Resources for bioinformatic tutorials](https://svenbuerki.github.io/EEB603_Reproducible_Science/index.html#Publis)
## 12                                                                                                                        [Chapter 3](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap3)
## 13                                                                                                                        [Chapter 4](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap4)
## 14                                                                                                              [Chapter 5 - Part A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partDM)
## 15                                                                                                             [Chapter 5 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#repcode)
## 16                                                                                                                        [Chapter 4](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap4)
## 17                                                                                                              [Chapter 5 - Part A](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#partDM)
## 18                                                                                                             [Chapter 5 - Part B](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#repcode)
## 19                            [Chapter 6](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap6) / [Chapter 7](https://svenbuerki.github.io/EEB603_Reproducible_Science/Chapters.html#chap7)
## 20                                                                                                                                                                                                                 
## 21                                                                                                                                                                                                                 
## 22                                                                                                                                                                                                                 
## 23                                                                                                                                                                                                                 
## 24                                                                                                                                                                                                                 
## 25                                                                                                                                                                                                                 
## 26                                                                                                                                                                                                                 
## 27                                                                                                                                                                                                                 
## 28                                                                                                                                                                                                                 
## 29                                                                                                                                                                                                                 
## 30                                                                                                                                                                                                                 
## 31                                                                                                                                                                                                                 
## 32                                                                                                                                                                                                                 
##    Completed
## 1          _
## 2          x
## 3          x
## 4          x
## 5          x
## 6          x
## 7          x
## 8          x
## 9          x
## 10         x
## 11         x
## 12         x
## 13         x
## 14         x
## 15         x
## 16         x
## 17         x
## 18         x
## 19         x
## 20          
## 21          
## 22          
## 23          
## 24          
## 25          
## 26          
## 27          
## 28          
## 29          
## 30          
## 31          
## 32

source_data() will always download the most recent version of the file from the master branch and return its unique SHA-1 hash. However, you could also download a prior version of the file by providing its specific SHA-1 hash associated to a previous commit.

8.8.5 Retrieving csv from previous commit

Retrieving the csv file associated to a specific commit can be done by applying the following approach (here using the same example than above):

  1. To find a file’s particular commit raw URL navigate to its location on GitHub by clicking here.
  2. Click the History button. This action will take you to a page listing all of the file’s versions (see Figure 8.6).
    Screenshot of the GitHub commit history page for Timetable_EEB603_topic_tasks.csv, showing four commits grouped by date, each row listing the commit message and its short SHA on the right, with Newer and Older paging buttons at the bottom.

    Figure 8.6: Page showing commit history associated to Timetable_EEB603_topic_tasks.csv.

  3. Click on the Browse the repository at this point in the history button next to the version of the file that you want to use (here Commits on Aug 23, 2021; Figure 8.6). This will take you to the version of the file at this point in history.
    Screenshot of the GitHub file view of an earlier version of Timetable_EEB603_topic_tasks.csv, selected in the branch dropdown, with the header noting the commit and the table below listing topics alongside their deadlines.

    Figure 8.7: Page showing prior commit associated to Timetable_EEB603_topic_tasks.csv.

  4. Clicking on the Raw button will load the csv file and allow you to retrieve the URL (as done above).
  5. Download the file using the following R code:
### ~~~ Store raw URL into object ~~~
urlcsvold <- "https://raw.githubusercontent.com/svenbuerki/EEB603_Reproducible_Science/1312eae07e0515d8d2423ff83834b184cf6eeb8d/Data/Timetable_EEB603_topic_tasks.csv"
### ~~~ Download/Import csv into R ~~~
csvTimeTableOld <- repmis::source_data(url = urlcsvold)
## Downloading data from: https://raw.githubusercontent.com/svenbuerki/EEB603_Reproducible_Science/1312eae07e0515d8d2423ff83834b184cf6eeb8d/Data/Timetable_EEB603_topic_tasks.csv
## SHA-1 hash of the downloaded data file is:
## e7071b1e85ada38d6b2cf2a93bbb43c2b96a331f
### ~~~ Check class ~~~ Class should be `data.frame`
class(csvTimeTableOld)
## [1] "data.frame"
### ~~~ Print csv ~~~
print(csvTimeTableOld)
##                                              Topic
## 1                                         Syllabus
## 2              Example of a bioinformatic tutorial
## 3                      Chap. 1 - R Markdown part A
## 4                      Chap. 1 - R Markdown part B
## 5                      Chap. 1 - R Markdown part C
## 6                 Chap. 1 - User-defined functions
## 7                                Chap. 1 - Wrap-up
## 8                                          Chap. 2
## 9                                          Chap. 3
## 10                        Chap. 4: Data management
## 11                      Chap. 4: Reproducible code
## 12                      Chap. 5: Getting published
## 13 Chap. 5: Writing papers in R Markdown (rticles)
## 14                                             TBD
## 15                     Bioinfo. tutorial - Chap. 6
## 16                     Bioinfo. tutorial - Chap. 6
## 17                     Bioinfo. tutorial - Chap. 7
## 18                     Bioinfo. tutorial - Chap. 7
## 19                     Bioinfo. tutorial - Chap. 8
## 20                     Bioinfo. tutorial - Chap. 8
## 21                     Bioinfo. tutorial - Chap. 9
## 22                     Bioinfo. tutorial - Chap. 9
## 23                    Bioinfo. tutorial - Chap. 10
## 24                    Bioinfo. tutorial - Chap. 10
## 25                    Bioinfo. tutorial - Chap. 11
## 26                    Bioinfo. tutorial - Chap. 11
## 27                       Individual projects - Q&A
## 28                              Oral presentations
## 29                              Oral presentations
## 30                              Oral presentations
##                                                       Task Deadline
## 1                                                                  
## 2                                                                  
## 3                                        Work on bioinfo. tutorials
## 4                                        Work on bioinfo. tutorials
## 5                                        Work on bioinfo. tutorials
## 6                                        Work on bioinfo. tutorials
## 7                                        Work on bioinfo. tutorials
## 8     Read Baker (2016) and prepare for discussing outcome of study
## 9                                        Work on bioinfo. tutorials
## 10                                       Work on bioinfo. tutorials
## 11                                       Work on bioinfo. tutorials
## 12                                       Work on bioinfo. tutorials
## 13          Turn in tutorial for Chap. 6 & start individual reports
## 14                       Upload tutorial of Chap. 6 on Google Drive
## 15                                     Turn in tutorial for Chap. 7
## 16                       Upload tutorial of Chap. 7 on Google Drive
## 17                                     Turn in tutorial for Chap. 8
## 18                       Upload tutorial of Chap. 8 on Google Drive
## 19                                     Turn in tutorial for Chap. 9
## 20                       Upload tutorial of Chap. 9 on Google Drive
## 21                                    Turn in tutorial for Chap. 10
## 22                      Upload tutorial of Chap. 10 on Google Drive
## 23                                    Turn in tutorial for Chap. 11
## 24                      Upload tutorial of Chap. 11 on Google Drive
## 25 Students work on ind. projects: Review literature (see Syllabus)
## 26         Students work on ind. projects: Data management workflow
## 27                                    Students work on ind. project
## 28                                    Work on reports/presentations
## 29                                             Turn in ind. reports
## 30
  1. You can see here that the SHA-1 hash is different from our previous download confirming that the files are different.

8.9 Downloading and processing GitHub repositories

Before being able to download a GitHub repository and work with its files into R, you have to find the URL pointing to the compressed zip file containing all the files for the target repository. In this section, we will demonstrate how to obtain this information by using the Sagebrush_rooting_in_vitro_prop GitHub repository. As mentioned above, we will be downloading the whole repository and then extract all the csv files in the 01_Raw_Data/ folder (see Figure 8.2).

8.9.1 How to retrieve the URL for the GitHub repository?

To retrieve the URL associated to the compressed zip file containing all files for the repository do the following:

  1. Navigate to the GitHub repository page by clicking here (see Figure 8.8).
    Screenshot of the GitHub code page for the Sagebrush_rooting_in_vitro_prop repository with the green Code button clicked, showing the Clone panel with the HTTPS URL and, over its Download ZIP entry, a browser right-click menu where Copy Link is highlighted.

    Figure 8.8: GitHub repository page for Sagebrush_rooting_in_vitro_prop.

  2. To copy the URL pointing to the compressed zip file do the following actions:
  • Click on the Code button.
  • Navigate to the Download ZIP section.
  • Right-click to copy the link (as shown in Figure 8.8).
  1. The copied URL (https://github.com/svenbuerki/Sagebrush_rooting_in_vitro_prop/archive/refs/heads/master.zip) will serve as input for downloading the repository on your local computer (see below).

8.9.2 Downloading GitHub repository on your computer

Now that we have secured the URL pointing to the compressed zip file for the target repository (by copying it), we will use this URL and the base R download.file() function as input to download the file on our local computer. Since compressed files could be large, we are also providing some code to check if the file already exists on your computer before downloading.

### ~~~ Store URL in object ~~~ Paste the URL that you
### copied in the previous section here
URLrepo <- "https://github.com/svenbuerki/Sagebrush_rooting_in_vitro_prop/archive/refs/heads/master.zip"
### ~~~ Download the repository from GitHub ~~~ Arguments:
### - url: URLrepo - destfile: Path and name of destination
### file on your computer YOU HAVE TO ADJUST PATH TO YOUR
### COMPUTER
# First check if the file exists, if yes then return file
# already downloaded else proceed with download
if (file.exists("Data/GitHubRepoSagebrush.zip") == TRUE) {
    # File already exists!
    print("file already exists and doesn't need to be downloaded!")
} else {
    # Download the file
    print("Downloading GitHub repository!")
    download.file(url = URLrepo, destfile = "Data/GitHubRepoSagebrush.zip")
}
## [1] "file already exists and doesn't need to be downloaded!"

8.9.3 Extracting raw data from compressed GitHub repository

Compressed files can be quite large and you might would like to avoid decompressing them, but rather accessing target files and only decompressing those. Here, we are practicing such approach by using GitHubRepoSagebrush.zip and targeting csv files in the 01_Raw_Data/ folder.

8.9.3.1 What is the size of the zip file?

To estimate the size (in bytes) of a file you can use base R function file.size() as follows:

### ~~~ Infer file size of GitHubRepoSagebrush.zip ~~~
# Transform and round file size from bytes to Mb
ZipSize <- round(file.size("Data/GitHubRepoSagebrush.zip")/1e+06,
    2)
print(paste("The zip file size is", ZipSize, "Mb", sep = " "))
## [1] "The zip file size is 22.26 Mb"

8.9.3.2 Decompressing and saving csv files in 01_Raw_Data/

Finally, we can now i) list all files in the zip file, ii) identify csv files in 01_Raw_Data/ and iii) save these files on our local computer in a folder entitled 01_Raw_Data/. These files will then constitute the raw data for your subsequent analyses.

### ~~~ List all files in zip file without decompressing it
### ~~~
filesZip <- as.character(unzip("Data/GitHubRepoSagebrush.zip",
    list = TRUE)$Name)
### ~~~ Identify files in 01_Raw_Data/ that are csv ~~~ Use
### grepl() to match criteria
targetF <- which(grepl("01_Raw_Data/", filesZip) & grepl(".csv",
    filesZip) == TRUE)
# Subset files from filesZip to only get our target files
rawcsvfiles <- filesZip[targetF]
# print list of target files
print(rawcsvfiles)
## [1] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/1_block_8_12_2020 - 1_block.csv"  
## [2] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/2_block_8_15_2020 - 2_block.csv"  
## [3] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/3_block_8_15_2020 - 3_block.csv"  
## [4] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/4_block_8_16_2020 - 4_block.csv"  
## [5] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/5_block_8_19_2020 - 5_block.csv"  
## [6] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/Phenotypes_sagebrush_in_vitro.csv"
## [7] "Sagebrush_rooting_in_vitro_prop-master/01_Raw_Data/Survival_height_clones.csv"
### ~~~ Create local directory to save csv files ~~~ Check
### if the folder already exists if not then creates it
output_dir <- file.path(paste0("Data/01_Raw_Data/"))
if (dir.exists(output_dir)) {
    print(paste0("Dir", output_dir, " already exists!"))
} else {
    print(paste0("Created ", output_dir))
    dir.create(output_dir)
}
## [1] "DirData/01_Raw_Data/ already exists!"
### ~~~ Save csv in output_dir ~~~ Use loop to read csv
### file in and then save it in output_dir
for (i in 1:length(rawcsvfiles)) {
    ### ~~~ Decompress and read in csv file ~~~
    tempcsv <- read.csv(unz("Data/GitHubRepoSagebrush.zip", rawcsvfiles[i]))
    ### ~~~ Write file in ~~~ Extract file name
    csvName <- strsplit(rawcsvfiles[i], split = "01_Raw_Data/")[[1]][2]
    # Write csv file in output_dir
    write.csv(tempcsv, file = paste0(output_dir, csvName))
}

We can verify that all the files are in the newly created directory on your computer by listing them as follows (compare your results with files shown in Figure 8.2):

# List all the files in output_dir (on your local computer)
list.files(paste0(output_dir))
## [1] "1_block_8_12_2020 - 1_block.csv"   "2_block_8_15_2020 - 2_block.csv"  
## [3] "3_block_8_15_2020 - 3_block.csv"   "4_block_8_16_2020 - 4_block.csv"  
## [5] "5_block_8_19_2020 - 5_block.csv"   "Phenotypes_sagebrush_in_vitro.csv"
## [7] "Survival_height_clones.csv"

9 References

Allaire, J., Y. Xie, C. Dervieux, R Foundation, H. Wickham, Journal of Statistical Software, R. Vaidyanathan, et al. 2024. Rticles: Article formats for r markdown. Available at: https://github.com/rstudio/rticles.
Allen, L., J. Scott, A. Brand, M. Hlava, and M. Altman. 2014. Publishing: Credit where credit is due. Nature 508: 312–313.
Baker, M. 2016. 1,500 scientists lift the lid on reproducibility. Nature 533: 452–454. Available at: https://doi.org/10.1038/533452a.
Barron, R., P. Martinez, M. Serpe, and S. Buerki. 2020. Development of an in vitro method of propagation for artemisia tridentata subsp. Tridentata to support genome sequencing and genotype-by-environment research. Plants 9: Available at: https://www.mdpi.com/2223-7747/9/12/1717.
British Ecological Society ed. 2014a. A guide to data management in ecology and evolution. British Ecological Society.
British Ecological Society ed. 2014b. A guide to getting published in ecology and evolution. British Ecological Society.
British Ecological Society ed. 2014c. A guide to reproducible code in ecology and evolution. British Ecological Society.
Cargill, M., and P. O’Connor. 2011. Writing scientific research articles: Strategy and steps. Wiley-Blackwell.
Carroll, S.R., E. Herczog, M. Hudson, K. Russell, and S. Stall. 2021. Operationalizing the CARE and FAIR principles for indigenous data futures. Scientific Data 8: 108. Available at: https://doi.org/10.1038/s41597-021-00892-0.
Fox, C.W., and C.S. Burns. 2015. The relationship between manuscript title structure and success: Editorial decisions and citation performance for an ecological journal. Ecology and Evolution 5: 1970–1980. Available at: https://onlinelibrary.wiley.com/doi/abs/10.1002/ece3.1480.
Gandrud, C. 2016. Repmis: Miscellaneous tools for reproducible research. Available at: https://CRAN.R-project.org/package=repmis.
Gandrud, C. 2015. Reproducible Research with R and RStudio. 2nd ed. C. Gandrud [ed.],. CRC Press.
Grant, S., K.S. Corker, D. Mellor, et al. 2026. TOP 2025: An update to the transparency and openness promotion guidelines. Research Integrity and Peer Review 11: 40. Available at: https://doi.org/10.1186/s41073-026-00223-0.
Groom, Q., L. Weatherdon, and I.R. Geijzendorffer. 2017. Is citizen science an open science in the case of biodiversity observations? Journal of Applied Ecology 54: 612–617. Available at: https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/1365-2664.12767.
Hester, J., G. Csárdi, H. Wickham, W. Chang, M. Morgan, and D. Tenenbaum. 2019. Remotes: R package installation from remote repositories, including GitHub. Available at: https://github.com/r-lib/remotes#readme.
Khoo, S.Y.-S. 2019. Article processing charge hyperinflation and price insensitivity: An open access sequel to the serials crisis. LIBER Quarterly: The Journal of the Association of European Research Libraries 29: 1–18. Available at: https://liberquarterly.eu/article/view/10729.
Munafo, M.R., B.A. Nosek, D.V.M. Bishop, K.S. Button, C.D. Chambers, N.P. du Sert, U. Simonsohn, et al. 2017. A manifesto for reproducible science. Nature human behaviour 1: 0021.
National Academies of Sciences, Engineering, and Medicine. 2019. Reproducibility and replicability in science. The National Academies Press, Washington, DC. Available at: https://nap.nationalacademies.org/catalog/25303/reproducibility-and-replicability-in-science.
Nosek, B.A., G. Alter, G.C. Banks, D. Borsboom, S.D. Bowman, S.J. Breckler, S. Buck, et al. 2015. Promoting an open research culture. Science 348: 1422–1425. Available at: http://science.sciencemag.org/content/348/6242/1422.
Nosek, B.A., C.R. Ebersole, A.C. DeHaven, and D.T. Mellor. 2018. The preregistration revolution. Proceedings of the National Academy of Sciences 115: 2600–2606. Available at: http://www.pnas.org/content/115/11/2600.
Nosek, B.A., J.R. Spies, and M. Motyl. 2012. Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7: 615–631. Available at: https://doi.org/10.1177/1745691612459058.
R Core Team. 2020. Foreign: Read data stored by ’minitab’, ’s’, ’SAS’, ’SPSS’, ’stata’, ’systat’, ’weka’, ’dBase’, ... Available at: https://CRAN.R-project.org/package=foreign.
R Core Team. 2019. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available at: https://www.R-project.org/.
RStudio Team. 2020. RStudio: Integrated development environment for r. RStudio, PBC., Boston, MA. Available at: http://www.rstudio.com/.
Smith, J.F., T.H. Parker, S. Nakagawa, J. Gurevitch, Ecology, and T.(Tools. for T. in Evolution) Working Group. 2016. Promoting transparency in evolutionary biology and ecology. Systematic Botany 41: 495–497. Available at: http://www.bioone.org/doi/abs/10.1600/036364416X692262.
Troudet, J., R. Vignes-Lebbe, P. Grandcolas, and F. Legendre. 2018. The increasing disconnection of primary biodiversity data from specimens: How does it happen and how to handle it? Systematic Biologysyy044. Available at: http://dx.doi.org/10.1093/sysbio/syy044.
Wagenknecht, K., T. Woods, F.G. Sanz, M. Gold, A. Bowser, S. Rüfenacht, L. Ceccaroni, and J. Piera. 2021. EU-Citizen.Science: A Platform for Mainstreaming Citizen Science and Open Science in Europe. Data Intelligence 3: 136–149. Available at: https://doi.org/10.1162/dint_a_00085.
Wickham, H. 2014. Advanced R. Taylor & Francis. Available at: https://books.google.com/books?id=PFHFNAEACAAJ.
Wickham, H., and J. Bryan. 2019. Readxl: Read excel files. Available at: https://CRAN.R-project.org/package=readxl.
Wilkinson, M.D., M. Dumontier, Ij.J. Aalbersberg, G. Appleton, M. Axton, A. Baak, N. Blomberg, et al. 2016. The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3: 160018. Available at: https://doi.org/10.1038/sdata.2016.18.
Xie, Y. 2025a. Bookdown: Authoring books and technical documents with r markdown. Available at: https://github.com/rstudio/bookdown.
Xie, Y. 2016. Bookdown: Authoring books and technical documents with R markdown. Chapman; Hall/CRC, Boca Raton, Florida. Available at: https://bookdown.org/yihui/bookdown.
Xie, Y. 2015. Dynamic documents with R and knitr. 2nd ed. Chapman; Hall/CRC, Boca Raton, Florida. Available at: https://yihui.org/knitr/.
Xie, Y. 2025b. Knitr: A general-purpose package for dynamic report generation in r. Available at: https://yihui.org/knitr/.
Xie, Y., J.J. Allaire, and G. Grolemund. 2018. R markdown: The definitive guide. Chapman; Hall/CRC, Boca Raton, Florida. Available at: https://bookdown.org/yihui/rmarkdown.

Appendices

A Appendix 1

Citations of all R packages used to generate this report.

[1] J. Allaire, Y. Xie, C. Dervieux, et al. rmarkdown: Dynamic Documents for R. R package version 2.30. 2025. https://github.com/rstudio/rmarkdown.

[2] J. Allaire, Y. Xie, C. Dervieux, et al. rticles: Article Formats for R Markdown. R package version 0.27. 2024. https://github.com/rstudio/rticles.

[3] S. M. Bache and H. Wickham. magrittr: A Forward-Pipe Operator for R. R package version 2.0.4. 2025. https://magrittr.tidyverse.org.

[4] C. Boettiger. knitcitations: Citations for Knitr Markdown Files. R package version 1.0.12. 2021. https://github.com/cboettig/knitcitations.

[5] J. Cheng, C. Sievert, B. Schloerke, et al. htmltools: Tools for HTML. R package version 0.5.9. 2025. https://github.com/rstudio/htmltools.

[6] R. Francois and D. Hernangómez. bibtex: Bibtex Parser. R package version 0.5.1. 2023. https://github.com/ropensci/bibtex.

[7] C. Glur. data.tree: General Purpose Hierarchical Data Structure. R package version 1.2.0. 2025. https://github.com/gluc/data.tree.

[8] R. Iannone and O. Roy. DiagrammeR: Graph/Network Visualization. R package version 1.0.11. 2024. https://rich-iannone.github.io/DiagrammeR/.

[9] M. C. Koohafkan. kfigr: Integrated Code Chunk Anchoring and Referencing for R Markdown Documents. R package version 1.2.1. 2021. https://github.com/mkoohafkan/kfigr.

[10] R. Lesur. klippy: Copy to Clipboard Buttons for R Markdown HTML Documents. R package version 0.0.0.9500. 2026. https://rlesur.github.io/klippy.

[11] Y. Qiu. prettydoc: Creating Pretty Documents from R Markdown. R package version 0.4.1. 2021. https://github.com/yixuan/prettydoc.

[12] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria, 2025. https://www.R-project.org/.

[13] K. Ren and K. Russell. formattable: Create Formattable Data Structures. R package version 0.2.1. 2021. https://renkun-ken.github.io/formattable/.

[14] H. Wickham, J. Bryan, M. Barrett, et al. usethis: Automate Package and Project Setup. R package version 3.2.1. 2025. https://usethis.r-lib.org.

[15] H. Wickham, R. François, L. Henry, et al. dplyr: A Grammar of Data Manipulation. R package version 1.2.1. 2026. https://dplyr.tidyverse.org.

[16] H. Wickham, J. Hester, W. Chang, et al. devtools: Tools to Make Developing R Packages Easier. R package version 2.4.6. 2025. https://devtools.r-lib.org/.

[17] Y. Xie. bookdown: Authoring Books and Technical Documents with R Markdown. Boca Raton, Florida: Chapman and Hall/CRC, 2016. ISBN: 978-1138700109. https://bookdown.org/yihui/bookdown.

[18] Y. Xie. bookdown: Authoring Books and Technical Documents with R Markdown. R package version 0.46. 2025. https://github.com/rstudio/bookdown.

[19] Y. Xie. Dynamic Documents with R and knitr. 2nd. ISBN 978-1498716963. Boca Raton, Florida: Chapman and Hall/CRC, 2015. https://yihui.org/knitr/.

[20] Y. Xie. “knitr: A Comprehensive Tool for Reproducible Research in R”. In: Implementing Reproducible Computational Research. Ed. by V. Stodden, F. Leisch and R. D. Peng. ISBN 978-1466561595. Chapman and Hall/CRC, 2014.

[21] Y. Xie. knitr: A General-Purpose Package for Dynamic Report Generation in R. R package version 1.50. 2025. https://yihui.org/knitr/.

[22] Y. Xie, J. Allaire, and G. Grolemund. R Markdown: The Definitive Guide. Boca Raton, Florida: Chapman and Hall/CRC, 2018. ISBN: 9781138359338. https://bookdown.org/yihui/rmarkdown.

[23] Y. Xie, J. Cheng, X. Tan, et al. DT: A Wrapper of the JavaScript Library DataTables. R package version 0.34.0. 2025. https://github.com/rstudio/DT.

[24] Y. Xie, C. Dervieux, and E. Riederer. R Markdown Cookbook. Boca Raton, Florida: Chapman and Hall/CRC, 2020. ISBN: 9780367563837. https://bookdown.org/yihui/rmarkdown-cookbook.

[25] H. Zhu. kableExtra: Construct Complex Table with kable and Pipe Syntax. R package version 1.4.0. 2024. http://haozhu233.github.io/kableExtra/.

B Appendix 2

Version information about R, the operating system (OS) and attached or R loaded packages. This appendix was generated using sessionInfo().

## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Sequoia 15.7.9
## 
## Matrix products: default
## BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
## 
## locale:
## [1] en_US/en_US/en_US/C/en_US/en_US
## 
## time zone: America/Chicago
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] repmis_0.5.1         klippy_0.0.0.9500    rticles_0.27        
##  [4] DiagrammeR_1.0.11    DT_0.34.0            data.tree_1.2.0     
##  [7] kfigr_1.2.1          devtools_2.4.6       usethis_3.2.1       
## [10] bibtex_0.5.1         knitcitations_1.0.12 htmltools_0.5.9     
## [13] prettydoc_0.4.1      magrittr_2.0.4       dplyr_1.2.1         
## [16] kableExtra_1.4.0     formattable_0.2.1    bookdown_0.46       
## [19] rmarkdown_2.30       knitr_1.50          
## 
## loaded via a namespace (and not attached):
##  [1] xfun_0.54          bslib_0.9.0        htmlwidgets_1.6.4  remotes_2.5.0     
##  [5] visNetwork_2.1.4   vctrs_0.7.3        tools_4.5.2        crosstalk_1.2.2   
##  [9] generics_0.1.4     curl_7.0.0         tibble_3.3.0       RefManageR_1.4.0  
## [13] R.oo_1.27.1        pkgconfig_2.0.3    data.table_1.17.8  RColorBrewer_1.1-3
## [17] assertthat_0.2.1   R.cache_0.17.0     lifecycle_1.0.5    compiler_4.5.2    
## [21] farver_2.1.2       stringr_1.6.0      textshaping_1.0.4  sass_0.4.10       
## [25] yaml_2.3.12        pillar_1.11.1      jquerylib_0.1.4    R.utils_2.13.0    
## [29] ellipsis_0.3.2     cachem_1.1.0       magick_2.9.0       sessioninfo_1.2.3 
## [33] tidyselect_1.2.1   digest_0.6.39      stringi_1.8.9      purrr_1.2.2       
## [37] fastmap_1.2.0      cli_3.6.5          pkgbuild_1.4.8     withr_3.0.2       
## [41] scales_1.4.0       backports_1.5.0    lubridate_1.9.4    timechange_0.3.0  
## [45] httr_1.4.7         R.methodsS3_1.8.2  memoise_2.0.1      evaluate_1.0.5    
## [49] viridisLite_0.4.2  rlang_1.3.0        Rcpp_1.1.0         glue_1.8.0        
## [53] formatR_1.14       xml2_1.5.1         pkgload_1.4.1      svglite_2.2.2     
## [57] rstudioapi_0.17.1  jsonlite_2.0.0     R6_2.6.1           plyr_1.8.9        
## [61] systemfonts_1.3.1  fs_1.6.6

  1. What is a Hypothesis? A hypothesis is a tentative, testable answer to a scientific question. Once a scientist has a scientific question they perform a literature review to find out what is already known on the topic. Then this information is used to form a tentative answer to the scientific question. Keep in mind, that the hypothesis also has to be testable since the next step is to do an experiment to determine whether or not the hypothesis is right! A hypothesis leads to one or more predictions that can be tested by experimenting. Predictions often take the shape of “If ____then ____” statements, but do not have to. Predictions should include both an independent variable (the factor you change in an experiment) and a dependent variable (the factor you observe or measure in an experiment). A single hypothesis can lead to multiple predictions.↩︎

  2. GBIF — the Global Biodiversity Information Facility — is an international network and research infrastructure funded by the world’s governments and aimed at providing anyone, anywhere, open access to data about all types of life on Earth.↩︎

  3. Postdiction involves explanation after the fact.↩︎