miyaku / blog

merging mastodon export outboxes

if you’re using slurp to import posts into a gotosocial instance from a mastodon account, maybe you have multiple mastodon exports from a single account that you want to merge together before importing (e.g. you have multiple exports over time and auto-delete turned on for space reasons on your old server, but you want your gotosocial to have most of your old posts)

here’s a ruby script i wrote to merge two mastodon export outbox.json files, invoke with e.g. ./mergeoutboxes.rb ~/archive-1/outbox.json ~/archive-2/outbox.json ~/archive-merge/outbox.json:

#!/usr/bin/env ruby

require 'json'

file_1 = JSON.parse(File.read(ARGV[0]))
file_2 = JSON.parse(File.read(ARGV[1]))

$stderr.puts "Combining #{file_1['totalItems']} + " +
             "#{file_2['totalItems']} " + 
             "(#{file_1['totalItems'] + file_2['totalItems']})..."
$stderr.puts "Actually #{file_1['orderedItems'].count} + " +
             "#{file_2['orderedItems'].count} " +
             "(#{file_1['orderedItems'].count + file_2['orderedItems'].count})..."
file_1['orderedItems'] += file_2['orderedItems']
file_1['orderedItems'].uniq! {|e| e['id']}
total_items = file_1['orderedItems'].count
$stderr.puts "Total unique items: #{total_items}"
file_1['totalItems'] = total_items

File.write(ARGV[2], JSON.dump(file_1))

you can then use e.g. rsync to create a merged media_attachments folder before importing with slurp