Skip to main content

Importing the Bible to Roam - Final Solution

In this post I will present my final solution for getting the Bible imported into Roam Research. I have finally found a way to drastically decrease the size of the imported Roam graph, but still have a reasonable solution for accessing multiple translations and cross references. I will share my thoughts on performance and potential next steps for making the Bible work even better in Roam. If you are interested in the topic, I have two other posts on the subject. Links are at the end.

  • You can download the Roam .EDN archive for the Bible from here: ASV.edn
    • For a real time demonstration of the full import process within 3 minutes check out this video.
    • WARNING: by restoring the ASV.edn into your graph, all other data in your graph will be lost!
  • The source code for the converter is here: Roam.json Bible importer with BUTTON for verse numbers v1.ipynb
    • Use this if the primary translation you'd like to use in Roam is not the American Standard Version. Also you can tweak how the verses are formatted and whatever else you might want to change on the import.
 

If you haven't watched my other video about the Bible in Roam, I recommend watching that also. There I briefly show how you can add your own notes into your Roam Bible, and how it can become a powerful Study-Bible where you'll be able to find your notes and reflections when you revisit a verse in the future. Follow this link: Roam Bible Tour - YouTube. Also be sure to read my other post as it explains some of the logic for how I intend this Bible to be used: Study Bible or ePub Books in Roam? My Roller Coaster Ride with Roam JSON

Roam is not ready for handling a large body of text

In its current state Roam does not seem to be ready for handling a large text-corpus. In my previous attempts with importing the Bible I tried loading 33 000 pages and 96 000 blocks into Roam. The loading process took days... and as I was reaching 70 000 blocks in the graph, the import speed dropped to well below one block per second. More concerningly the graph took minutes to open and was very sluggish to work with even on a reasonably powerful desktop PC. Access from a tablet or phone were simply out of the question.

Based on this experiment, my conclusion is not to load full-text books into my main graph. I also advise keeping the Bible in a separate Study-Bible Roam graph, ring fenced from your other notes.

Additionally, the local Roam graph is distinctively slower than online Roam. While this version of the Bible works well in an online Roam database, working in a local graph I ran into performance problems.

Scaling down the size of the Bible graph

I did two things to drastically reduce the size of the Bible graph.

1. No pages for verses

I decided against creating a page for each verse. e.g. [[Genesis 1:1]]. This approach seemed convenient for linking multiple translations of the same verse and to store cross references, but also accounted for ~32000 pages and ~32000 blocks in the graph. Since I am only importing a single translation this is not need. 

2. No cross references

Cross references created enormous number of backlinks in the database and also accounted for an additional ~32000 blocks. Cross references are also not crucial to the Bible-study workflow in Roam. Also, there are plenty of powerful search tools on the internet with extensive Bible cross references. Where Roam shines is not the cross referencing of the text-corpus but the cross referencing of your own notes built around that text. 

Lean approach to accessing other translations, searchability and more...

I believe the ability to search for a verse is important. You want to be able to enter "MAT 1:1" and find Matthew chapter 1 verse 1. Easy access to different translations of a verse would still be preferred as well as access to cross references.

I solved this challenge by adding the verse reference for each verse as a custom Roam button instead of a page link to the verse: {{1:BIBLE:MAT 1:1}}. This approach provides almost unlimited flexibility by putting what ever logic you may want into a {{[[roam/js]]}} script. Also the text in the button is indexed by Roam, thus entering "MAT 1:1" in the search bar yields Matthew 1:1.

For now, I created a simple event handler that opens up bible.com at the verse specified in the button. This script can be easily further customized to pull up cross references either from a file on the internet using roam/js, or to call an internet service for cross references. Your imagination and coding skills are really the only limit.

Here's my script. Note Roam42 is a prerequisite. Also the script is largely based on code I took from Roam42. Thanks @RoamHacker! You need to put this code into a code-block nested under {{[[roam/js]]}}.

window.bible = {
  async event(e) { 
    if(e.target.tagName=='BUTTON') {
      var block = e.target.closest('.roam-block');
      if (!block) {
        return;
      }
      var blockInfo = (await roam42.common.getBlockInfoByUID(block.id.substring( block.id.length -9)))[0][0].string;
      if(blockInfo.includes(e.target.textContent + ':BIBLE:' )) {
        const regex = '{{\\d+:BIBLE:(.{3})\\s(\\d+).(\\d+)}}';
        var v = blockInfo.match(regex);
        if(v.length == 4) {
          window.open('https://www.bible.com/bible/547/'+v[1]+'.'+v[2]+'.'+v[3]+'.KJVAE','_blank');
        }
      }
    }
  }
}

document.addEventListener('click', window.bible.event);

Import range of verses into your Daily Notes using Roam42 SmartBlocks


The Verses SmartBlock is already included in the ASV.edn available for download above. If you are doing your own import using JSON then you can use the SmartBlock available on GitHub: Import range of Bible verses as block references · Issue #159 · roamhacker/SmartBlocks

More about Roam

Like this post?
Show your support.

Comments

  1. Hi am interested in getting the NIV version of the bible into my Roam rather then the ASV one. Is this something you could assist me with. Amazing stuff by the way. The work you have done to get the entire bible into Roam looks immense. I am not that confident with script etc so hopefully you can put me on the right path. Thanks

    ReplyDelete
    Replies
    1. Hi Jesse,

      would be happy to help, however as much as I can see the NIV version is not in the public domain. I tried Googleing it, but I always get back to the 10 translations available in the biblehub file.
      If one of the 10 works for you, that is easy to generate - I can also load it for you... though it takes a few hours.
      0. King James Bible
      1. American Standard Version
      2. Douay-Rheims Bible
      3. Darby Bible Translation
      4. English Revised Version
      5. Webster Bible Translation
      6. World English Bible
      7. Young's Literal Translation
      8. American King James Version
      9. Weymouth New Testament

      If you have an epub version of the NIV you can try the epub converter (https://www.zsolt.blog/2020/12/read-books-in-roam-detailed-how-to.html) Because of the size of the Bible, the ePUB converter will need a slight modification to cut the .json files into smaller pieces so Roam can handle the import - I am happy to help with the change if you've got the .epub.

      Delete
  2. Amazing thanks so much for getting back to me. I think I have found an ePub version. I will try and track down your email to send that to you.

    ReplyDelete
  3. Replies
    1. You can DM me on Twitter @zsviczian https://twitter.com/zsviczian
      Also the info@zsolt.blog should work as well. It filters emails with .zip and other potentially harmful attachments. Best is if you send a link to your ePub on either of icloud, google drive, dropbox, onedrive, etc.

      Delete
  4. Hi Zsolt - I use Roam Research for my research - but hadn't though of using for bible scriptures - what a great project. Unfortunately, I am rather naĆÆve when it comes to the techy - programme stuff part of your approach - in particular the creation of edn format. I see you offer help with this. I would like to use the ESV - which is on BibleHub. Are you able to help.? Russell

    ReplyDelete

Post a Comment

Popular posts from this blog

Showcasing Excalidraw

Conor ( @Conaw ) pointed me to Excalidraw last week, and I was blown away by the tool and especially about the opportunities it opens up for  Roam Research ! It is a full-featured, embeddable sketching component ready for web integration. This post will showcase key Excalidraw features and discusses some of the issues I still need to solve to complete its integration into Roam. I spent most of my free time during the week integrating Excalidraw into Roam. This article will introduce Excalidraw by showcasing its features.

Mind mapping with Excalidraw in Obsidian

Mind-mapping is a powerful tool. In this post I will show you how and when I mindmap with Excalidraw in Obsidian and why mindmapping is such a good tool for Personal Knowledge Management. Like this post? Show your support.

Evergreen Note on Note-taking Strategies and Their Practical Implementations

This is an evergreen note that I will be revisit regularly to develop a comprehensive list of note-taking approaches including their practical implementations in various software tools. This post will serve as an elaborate table of contents, including a brief introductory discussion on the importance of note-taking, followed by a high-level walkthrough of each method. Links to posts and videos with detailed examples and descriptions will follow over the coming weeks and months.

Deep Dive Into Roam's Data Structure - Why Roam is Much More Than a Note Taking App

Which are the longest paragraphs in your graph? Which pages did you edit or create last week? How many paragraphs of text do you have in your database in total? Which pages do you have under a given namesapece (e.g. meetings/)?

contact: info@zsolt.blog