Skip to main content

Importing the Bible to Roam - Final Solution

In this post I will present my final solution for getting the Bible imported into Roam Research. I have finally found a way to drastically decrease the size of the imported Roam graph, but still have a reasonable solution for accessing multiple translations and cross references. I will share my thoughts on performance and potential next steps for making the Bible work even better in Roam. If you are interested in the topic, I have two other posts on the subject. Links are at the end.

  • You can download the Roam .EDN archive for the Bible from here: ASV.edn
    • For a real time demonstration of the full import process within 3 minutes check out this video.
    • WARNING: by restoring the ASV.edn into your graph, all other data in your graph will be lost!
  • The source code for the converter is here: Roam.json Bible importer with BUTTON for verse numbers v1.ipynb
    • Use this if the primary translation you'd like to use in Roam is not the American Standard Version. Also you can tweak how the verses are formatted and whatever else you might want to change on the import.
 

If you haven't watched my other video about the Bible in Roam, I recommend watching that also. There I briefly show how you can add your own notes into your Roam Bible, and how it can become a powerful Study-Bible where you'll be able to find your notes and reflections when you revisit a verse in the future. Follow this link: Roam Bible Tour - YouTube. Also be sure to read my other post as it explains some of the logic for how I intend this Bible to be used: Study Bible or ePub Books in Roam? My Roller Coaster Ride with Roam JSON

Roam is not ready for handling a large body of text

In its current state Roam does not seem to be ready for handling a large text-corpus. In my previous attempts with importing the Bible I tried loading 33 000 pages and 96 000 blocks into Roam. The loading process took days... and as I was reaching 70 000 blocks in the graph, the import speed dropped to well below one block per second. More concerningly the graph took minutes to open and was very sluggish to work with even on a reasonably powerful desktop PC. Access from a tablet or phone were simply out of the question.

Based on this experiment, my conclusion is not to load full-text books into my main graph. I also advise keeping the Bible in a separate Study-Bible Roam graph, ring fenced from your other notes.

Additionally, the local Roam graph is distinctively slower than online Roam. While this version of the Bible works well in an online Roam database, working in a local graph I ran into performance problems.

Scaling down the size of the Bible graph

I did two things to drastically reduce the size of the Bible graph.

1. No pages for verses

I decided against creating a page for each verse. e.g. [[Genesis 1:1]]. This approach seemed convenient for linking multiple translations of the same verse and to store cross references, but also accounted for ~32000 pages and ~32000 blocks in the graph. Since I am only importing a single translation this is not need. 

2. No cross references

Cross references created enormous number of backlinks in the database and also accounted for an additional ~32000 blocks. Cross references are also not crucial to the Bible-study workflow in Roam. Also, there are plenty of powerful search tools on the internet with extensive Bible cross references. Where Roam shines is not the cross referencing of the text-corpus but the cross referencing of your own notes built around that text. 

Lean approach to accessing other translations, searchability and more...

I believe the ability to search for a verse is important. You want to be able to enter "MAT 1:1" and find Matthew chapter 1 verse 1. Easy access to different translations of a verse would still be preferred as well as access to cross references.

I solved this challenge by adding the verse reference for each verse as a custom Roam button instead of a page link to the verse: {{1:BIBLE:MAT 1:1}}. This approach provides almost unlimited flexibility by putting what ever logic you may want into a {{[[roam/js]]}} script. Also the text in the button is indexed by Roam, thus entering "MAT 1:1" in the search bar yields Matthew 1:1.

For now, I created a simple event handler that opens up bible.com at the verse specified in the button. This script can be easily further customized to pull up cross references either from a file on the internet using roam/js, or to call an internet service for cross references. Your imagination and coding skills are really the only limit.

Here's my script. Note Roam42 is a prerequisite. Also the script is largely based on code I took from Roam42. Thanks @RoamHacker! You need to put this code into a code-block nested under {{[[roam/js]]}}.

window.bible = {
  async event(e) { 
    if(e.target.tagName=='BUTTON') {
      var block = e.target.closest('.roam-block');
      if (!block) {
        return;
      }
      var blockInfo = (await roam42.common.getBlockInfoByUID(block.id.substring( block.id.length -9)))[0][0].string;
      if(blockInfo.includes(e.target.textContent + ':BIBLE:' )) {
        const regex = '{{\\d+:BIBLE:(.{3})\\s(\\d+).(\\d+)}}';
        var v = blockInfo.match(regex);
        if(v.length == 4) {
          window.open('https://www.bible.com/bible/547/'+v[1]+'.'+v[2]+'.'+v[3]+'.KJVAE','_blank');
        }
      }
    }
  }
}

document.addEventListener('click', window.bible.event);

Import range of verses into your Daily Notes using Roam42 SmartBlocks


The Verses SmartBlock is already included in the ASV.edn available for download above. If you are doing your own import using JSON then you can use the SmartBlock available on GitHub: Import range of Bible verses as block references · Issue #159 · roamhacker/SmartBlocks

More about Roam

Like this post?
Show your support.

Comments

  1. Hi am interested in getting the NIV version of the bible into my Roam rather then the ASV one. Is this something you could assist me with. Amazing stuff by the way. The work you have done to get the entire bible into Roam looks immense. I am not that confident with script etc so hopefully you can put me on the right path. Thanks

    ReplyDelete
    Replies
    1. Hi Jesse,

      would be happy to help, however as much as I can see the NIV version is not in the public domain. I tried Googleing it, but I always get back to the 10 translations available in the biblehub file.
      If one of the 10 works for you, that is easy to generate - I can also load it for you... though it takes a few hours.
      0. King James Bible
      1. American Standard Version
      2. Douay-Rheims Bible
      3. Darby Bible Translation
      4. English Revised Version
      5. Webster Bible Translation
      6. World English Bible
      7. Young's Literal Translation
      8. American King James Version
      9. Weymouth New Testament

      If you have an epub version of the NIV you can try the epub converter (https://www.zsolt.blog/2020/12/read-books-in-roam-detailed-how-to.html) Because of the size of the Bible, the ePUB converter will need a slight modification to cut the .json files into smaller pieces so Roam can handle the import - I am happy to help with the change if you've got the .epub.

      Delete
  2. Amazing thanks so much for getting back to me. I think I have found an ePub version. I will try and track down your email to send that to you.

    ReplyDelete
  3. Replies
    1. You can DM me on Twitter @zsviczian https://twitter.com/zsviczian
      Also the info@zsolt.blog should work as well. It filters emails with .zip and other potentially harmful attachments. Best is if you send a link to your ePub on either of icloud, google drive, dropbox, onedrive, etc.

      Delete

Post a Comment

Popular posts from this blog

Deep Dive Into Roam's Data Structure - Why Roam is Much More Than a Note Taking App

Which are the longest paragraphs in your graph? Which pages did you edit or create last week? How many paragraphs of text do you have in your database in total? Which pages do you have under a given namesapece (e.g. meetings/)?

Showcasing Excalidraw

Conor ( @Conaw ) pointed me to Excalidraw last week, and I was blown away by the tool and especially about the opportunities it opens up for  Roam Research ! It is a full-featured, embeddable sketching component ready for web integration. This post will showcase key Excalidraw features and discusses some of the issues I still need to solve to complete its integration into Roam. I spent most of my free time during the week integrating Excalidraw into Roam. This article will introduce Excalidraw by showcasing its features.

Roam-Excalidraw Plugin MVP Release

  I am releasing the MVP version of the Roam-Excalidraw Plugin. Over the past two weeks, I have been super focused on getting to this point. As a consequence, this post is going to be shorter and more utilitarian than usual. I had to make a choice whether to release the plugin this weekend or to write a detailed blog post. I opted for the first.

My GTD - How I Organize Meetings and TODOs in Roam

How efficient is your workflow for keeping on top of all your meeting notes, action items, contacts, projects and more?  If you were to bump into someone unexpectedly would you be able to remind yourself of all the relevant topics you wanted to discuss with the person?  Can you remember all the things you wanted to get done when running your errands?  Can you keep track of your discussions with all the people you talk to regularly? In this post I will walk you through my meetings-actions-people workflow in Roam. If you are new to Roam and Roam42... Just in case you are not familiar with Roam , it is an ultra flexible note taking tool. It's like the Excel for text. If you want to find out more, there is tremendous amount of quality content available on YouTube, just search from "Roam Research". Equally, you can head over to RoamBrain.com for all the best links and more. My workf

contact: info@zsolt.blog