TumblThree - A Tumblr Backup Application

TumblThree - A Tumblr Backup Application

TumblThree is the code rewrite of TumblTwo, a free and open source Tumblr blog backup application, using C# with WPF and the MVVM pattern. It uses the Win Application Framework (WAF). It downloads photo, video, audio and text posts from a given tumblr blog.

Screenshots:

TumblThree - A Tumblr Backup Application.</a></p>
<p><a id=

Features:

  • Source code at github (Written in C# using WPF and MVVM).
  • Multiple concurrent downloads of a single blog.
  • Multiple concurrent downloads of different blogs.
  • Internationalization support (currently available: en, zh, ru, de, fr).
  • A download queue.
  • Autosave of the queuelist.
  • Save, clear and restore the queuelist.
  • A clipboard monitor that detects blogname.tumblr.com urls in the clipboard (copy and paste) and automatically adds the blog to the bloglist.
  • A settings panel (change download location, turn preview off/on, define number of concurrent downloads, set the imagesize of downloaded pictures, set download defaults, enable portable mode, etc.).
  • Uses Windows proxy settings.
  • A bandwidth throttler.
  • An option to download an url list instead of the actual files.
  • Set a start time for a automatic download (e.g. during nights).
  • An option to skip the download of a file if it has already been downloaded before in any currently added blog.
  • Uses SSL connections.
  • Preview of photos & videos.
  • Taskbar buttons and key bindings.

Blog backup/download:

  • Download of photo, video (only tumblr.com hosted), text, audio, quote, conversation, link and question posts.
  • Download meta information for photo, video and audio posts.
  • Downloads inlined photos and videos (e.g. photos embedded in question&answer posts).
  • Download of _raw image files (original/higher resolution pictures).
  • Support for downloading Imgur, Gfycat, Webmshare, Mixtape, Lolisafe, Uguu, Catbox and SafeMoe linked files in tumblr posts.
  • Download of safe mode/NSFW blogs.
  • Allows to download only original content of the blog and skip reblogged posts.
  • Can download only tagged posts.
  • Can download only specific blog pages instead of the whole blog.
  • Allows to download blog posts in a defined time span.
  • Can download hidden blogs (login required / dash board blogs).
  • Can download password protected blogs (of non-hidden blogs).

Liked/by backup/download:

  • A downloader for downloading "liked by" photos and videos instead of a tumblr blog (e.g. https://www.tumblr.com/liked/by/wallpaperfx/) (login required).
  • Download of _raw image files (original/higher resolution pictures).
  • Allows to download posts in a defined time span.

Tumblr search backup/download:

  • A downloader for downloading photos and videos from the tumblr search (e.g. http://www.tumblr.com/search/my+keywords).
  • Download of _raw image files (original/higher resolution pictures).
  • Can download only specific blog pages instead of the whole blog.

Tumblr tag search backup/download:

  • A downloader for downloading photos and videos from the tumblr tag search (e.g. http://www.tumblr.com/tagged/my+keywords) (login required).
  • Download of _raw image files (original/higher resolution pictures).
  • Allows to download posts in a defined time span.

Program Usage:

  • Extract the .zip file and run the application by double clicking TumblThree.exe.
  • Copy the url of any tumblr.com blog you want to backup from into the textbox at the bottom left. Afterwards, click on 'Add Blog' on the right side of it.
  • Alternatively, if you copy (ctrl-c) a tumblr.com blog url from the address bar/text file, the clipboard monitor from TumblThree will detect it and automatically add the blog.
  • To start the download process, click on 'Crawl'. The application will regularly check for (new) blogs in the queue and start processing them, until you stop the application by pressing 'Stop'. So, you can either add blogs to the queue via 'Add to Queue' or double click/drag'n'drop first and then click 'Crawl', or you start the download process first and add blogs to the queue afterwards.
  • A light blue bar left to the blog in the queue indicates a actively downloading blog.
  • The blog manager on the left side also indicates the state of each blog. A red background shows an offline blog, a green background an actively crawling blog and a purple background an enqueued blog.
  • You change the download location, the number of concurrent connections, the default backup settings for each newly added blog and various other settings in the 'Settings'.
  • In the Details window you can view statistics of your blog and set blog specific options. You can here what kind of post type (photo, video, audio, text, conversation, quote, link) to download.
  • For downloading only tagged posts, you'll have to do some steps:
    1. Add the blog url.
    2. Open the blog in the details tab, enter the tags in the Tags textbox in a comma separated list without the leading hash (#) sign. E.g. great big car,bears would search for images that are tagged for either a great big car or bears or both.
  • For downloading password protected blogs, you'll have to do some steps:
    1. Add the blog url.
    2. Open the blog in the details tab, enter the password in the Password textbox.
  • For downloading hidden blogs (login required blogs), you have to do some steps:
    1. Go to Settings, click on the Connection tab and fill in your tumblr email address (login) and password, then click the Authenticate button. If the login was successfully, the label will change and display your email address. The email address and password are not stored locally on disk but cookies are generated and saved in %LOCALAPPDATA%\TumblThree in json format.
    2. Add the blog url.
  • For downloading liked photos and videos, you'll have to do some steps:
    1. Go to Settings, click on the Connection tab and fill in your tumblr email address (login) and password, then click the Authenticate button. If the login was successfully, the label will change and display your email address. The email address and password are not stored locally on disk but cookies are generated and saved in %LOCALAPPDATA%\TumblThree in json format.
    2. Add the blog url including the liked/by string in the url (e.g. https://www.tumblr.com/liked/by/wallpaperfx/).
    3. For downloading your own likes, make sure you've (temporarily) enabled the following options in your blogs settings (i.e. https://www.tumblr.com/settings/blog/yourblogname):
      1. Likes -> Share posts you like (to enable the publicly visible liked/by page)
      2. Visibility -> blog is explicit (to see/download NSFW likes)
  • For downloading photos and videos from the tumblr search, you'll have to do some steps:
    1. Add the search url including your key words separated by plus signs (+) in the url (e.g. https://www.tumblr.com/search/my+special+tags).
  • For downloading photos and videos from the tumblr tag search, you'll have to do some steps:
    1. Go to Settings, click on the Connection tab and fill in your tumblr email address (login) and password, then click the Authenticate button. If the login was successfully, the label will change and display your email address. The email address and password are not stored locally on disk but cookies are generated and saved in %LOCALAPPDATA%\TumblThree in json format.
    2. Add the search url including your tags separated by plus signs (+) in the url (e.g. https://www.tumblr.com/tagged/my+special+tags).

Key Mappings:

  • Currently mapped keys:
    • double click on a blog adds it to the queue
    • drag and drop of blogs from the manager (left side) to the queue
    • space -- start crawl
    • ctrl-space -- pause crawl
    • shift-space -- stop crawl
    • del -- remove blog from queuelist
    • shift-del -- remove blog from blogmanager
    • ctrl-shift-g -- manually trigger the garbage collection

Getting Started:

The default settings should cover most users. You should only have to change the download location and the kind of posts you want to download. For this, in the Settings (click on the Settings button in the lower panel of the main user interface) you might want to change:

  • General -> Download location: Specifies where to download the files. The default is in a folder Blogs relative to the TumblThree.exe
  • Blog -> Settings applied to each blog upon addition:
    • Here you can set what posts newly added blogs will download per default. To change what each blog downloads, click on a blog in the main interface, select the Details Tab on the right and change the settings. This separation allows to download different kind of post for different blogs. You can change the download settings for multiple existing blogs by selecting them with shift+left click for a range or ctrl-a for all of them.
    • Note: You might want to always select:
      • Download Reblogged posts: Downloads reblogs, not just original content of the blog author.

Settings you might want to change if the download speed is not satisfactory:

  • Connection -> Concurrent connections: Specifies the number of connections used for downloading posts. The number is shared between all actively downloading blogs.
  • Connection -> Concurrent video connections: Specifies the number of connections used for downloading tumblr video posts. The vt.tumblr.com host regularly closes connections if the number is too high. Thus, the maximum number of vt.tumblr.com connections can be specified here independently.
  • Connection -> Concurrent blogs: Number of blogs to download in parallel.

Most likely you don't have to change any of the other connection settings. In particular, settings you should never change, unless you're sure you know what you are doing:

  • Connection -> Limit Tumblr Api Connections: Leave this checkbox checked and do not change the corresponding values of 90 connections per 60 seconds. If you still change them, you might end up with offline blogs or missing downloads.

Further Insights:

  • Note: All the following files are stored in json format and can be opened in any editor.
  • Application settings are stored in C:\Users\Username\AppData\Local\TumblThree\.
  • You can use the portable mode (settings->general) to stores the application settings in the same folder as the executable.
  • For each blog there is also a database (serialized class) file in the Index folder of the download location named after the blogname.tumblr. Here blog relative information is stored like what files have been downloaded, the url of the blog and when it was added. This allows you to move your downloaded files (photos, videos, audio files) to a different location without interfering with the download process.
  • Some settings aren't hooked up to the graphical user interface. It's possible to view all TumblThree settings by opening the settings.json in any editor located in C:\Users\Username\AppData\Local\TumblThree\. Their names should be self explainatory. Some notable settings to further fine tune the application include:
    • BufferSize: Allows to set the buffer size for downloading binary files (photos, videos) in multiples of 4KB. The default is 2MB, thus the BufferSize has a value of 512. Increasing this value reduces disk fragmentation as more of the file is kept in the memory before it gets written out to the disk but increases the memory usage.
    • MaxNumberOfRetries: Sets the maximum number of retries if a tumblr server forcefully closes the connection. This might regularly happen on the tumblr video host (vt.tumblr.com) if too many connections were opened in parallel. After the limit is exhausted, the file is left truncated, but is also not registered as a successful downloaded. Thus, the file can be resumed in the next crawl.
    • TumblrHosts: Contains a list of hosts which is tried for downloading _raw photos if the photo size is set to raw. If none of the hosts contains the _raw version, the actually scanned host is tried with the next lower resolution (1028).

Changelog:

2018-07-05:

  • Implements the Tumblr login process and cookie handling in code instead of relying on the Internet Explorer for the Tumblr login process.

2018-06-09:

  • Fixes hidden Tumblr blog download problems caused by the new Tumblr ToS.

2018-05-20:

  • Programmatically agrees to new ToS and GDPR.
  • Implements SVC authentication changes. The SVC service is used to display the dash board blogs (i.e. hidden tumblr blogs). Changes in this internal Tumblr api prohibited TumblThrees access.
  • Saves the last post id in successful hidden tumblr downloads.
  • Improves the text parser of the tumblr api and tumblr svc data models. Separated the slug from the url as the data models are inconsistent. Separated the photoset urls from the photo urls. Moved the date information into a separate column.
  • Minor text changes of some user interface elements.

2018-04-18:

  • Updates the tumblr blog crawler and the hidden tumblr datamodel to reflect tumblr api changes that break blog download of previous TumblThree versions.

2018-02-28:

  • Allows to download only specific pages of hidden Tumblr blogs and in the tumblr search.
  • Improves the proxy settings. TumblThree now uses the default Windows (Internet Explorer) settings if not overridden within TumblThree.
  • Changes the behavior of the timeout value (Settings->Connection->Timeout). The timeout value now counts file chunks of 4kb instead of the whole file download, thus it should better detect if a download is stalled or a connection dropped without canceling active downloads of larger files (e.g. videos).
  • Changes default timeout value (for new users) from 600s to 30s.
  • Fixes possible download of the same photo but with different resolutions. This happened if the _raw file download was interrupted (the timeout hit), then the same photo was queued for download with the _1280 resolution. If the blog was then subsequently queued again, the _raw file was downloaded next to the _1280 file.
  • Fixes reblog/original post detection in the tumblr hidden crawler.
  • Fixes check blog status during startup-option.
  • Fixes download of password protected tumblr blogs.
  • Adds Mixtape, Lolisafe, Uguu, Catbox and SafeMoe parser (thanks to bun-dev).

2017-12-31:

  • Fixes a bug that released the video connection semaphore too often. That means the slider in the settings for limiting the video downloads didn't work at all. It should properly limit the connections to the vt.tumblr.com host and prevent incomplete video downloads now.
  • Includes a rewrite of the blog detection during blog addition. It should reduce latency if you mass add blogs by copying urls into the clipboard (ctrl-c). Offline blogs aren't added anymore.
  • Notifies the user when a connection timeout has occurred. The message states whether the timeout has occurred during downloading or crawling. If it happened during crawling, you might want to re-queue the blog at some point to grab missing posts. A connection timeout should only happen if your connection is wonky. You can decrease/increase the timeout in the settings (settings->connection).
  • You can now specify in the Details-panel for each blog where its files should be downloaded. If the text box control is empty, the files are downloaded as in previous releases in the folder specified in the global download location (settings->general), plus the blogs name.
  • Imgur.com linked albums in tumblr posts are now entirely downloaded if enabled (details panel->external->download imgur). Previously, only directly linked images were detected.
  • Adds an option to load all blog databases into memory and compare each to-download binary file to all databases across TumblThree before downloading. If the file has already been downloaded in any blog before, the file is skipped and will not be counted as downloaded. You can enable this in the settings (settings->global).
  • Allows to add hidden tumblr blogs using the dashboard url (i.e. https://www.tumblr.com/dashboard/blog/blogtobackup).
  • Allows to add all blog types without the protocol suffix (i.e. wallpaperfx.tumblr.com, www.tumblr.com/search/cars).
  • Adds an option to enable a confirmation dialog before removing blogs (#186, #130, #98). It's off by default.

2017-11-17:

  • Adds support for downloading Imgur.com, Gfycat.com and Webmshare.com linked files in tumblr posts.
  • Improves downloading of tumblr liked/by photos and videos.

2017-10-20:

  • Restores bandwidth limiter functionality.

2017-10-13:

  • Changes the default _raw photo host.

2017-10-09:

  • Fixes crawler stop in hidden tumblr blog downloads.
  • Adds options to set the default blog settings for the download from time, download to time and tags in the settings menu.
  • Adds some (ar, el, es, fa, fi, he, hi, it, ja, ko, no, pa, pl, pt, th, tr and vi) google translate translations.

2017-09-08:

  • Can download password protected blogs of non-hidden blogs.
  • Minor UI updates.

2017-08-22:

2017-08-21:

  • French, Spanish and simplified Chinese translations.
  • Removes user interface lag during blog addition.
  • Allows to set the buffer size for downloading binary files in the settings.json in multiples of 4KB. The variable is called BufferSize. The new default is 2MB, thus the BufferSize has a value of 512. Previously it was set to 4KB, but apparently Windows does not do any useful caching on NTFS if multiple writes are concurrent and async. Thus, this should reduce disk fragmentation.
  • Uses .NET Framework 4.6 now as it should be available for all supported windows versions (Windows Vista and above).
  • Improved the selection handling in the details panel. If multiple blogs are selected, old values are now kept if they are the same for all blogs and changes are immediately reflected.
  • Audio file download support for tumblr and hidden tumblr blogs.
  • More code Refactoring.

2017-07-03:

  • Can download hidden (login required/dash board) blogs.

2017-06-30:

  • Improved performance and bugfixes.

2017-06-20:

  • Downloads high resolution (_raw) images.
  • Updated translations (German and Russian).
  • Applies changed settings immediately.

2017-06-04:

  • Sets the date modified date in the Explorer to the posts time.
  • Allows to download single or ranges of blog pages.
  • Full screen media preview.

2017-05-20:

  • Option to skip reblogged posts.
  • Improves detection of inlined photos and videos in text posts (e.g. in answer posts).

2017-05-14:

  • Portable mode.
  • Downloads liked photos and videos.

2017-04-18:

  • Code refactoring.
  • Uses async/await in most of the code instead of tasks from the threadpool.
  • Uses a consumer producer pattern for grabbing and downloading as the Tumblr api v1 is now rate limited.
  • Downloads are now resumable.
  • Data files are now saved as json instead of binary.
  • Reduced memory usage by layering off the downloaded file list and only load it if needed.
  • Improves ui responsiveness.

2017-01-08:

  • Improves the speed of the network code.
  • Adds an option to use a http proxy.
  • Downloads inline images of tumblr posts.
  • Added Russian translation.

2016-12-13:

  • Improves the ui scaling of the main window for smaller resolutions.
  • Prevents crawling of offline blogs.
  • If the same blog is multiple times in the queue and already once active, any other free crawler task will skip and remove any already active blog and proceed to the next inactive blog in the queue.
  • Improved german translation.

2016-12-10:

  • The check for already downloaded files is now independent from the actual host and based entirely on the filename. It look likes the host/mirror does actual vary which would result in a reload of the file since its url changed.
  • Add scrollbars to the settings window if the controls do not fit.
  • Safely replaces blog indexes. If there is an error (e.g. no disk space left) during the update of the index file, the old state should not be corrupted anymore.
  • Changes some color and adds an alternate color for the blog manager.

2016-11-23:

  • Fixes application crashes which occurred by adding tumblr blogs without title or description.
  • Decreases determination time of already downloaded files for large blogs (>100,000 posts) by at least three orders of magnitude.

2016-11-22:

  • Creates more meta information (post id, reblog key, timestamp, tags, slug, title) of the posts, including image, video and audio types.
  • Fixes the progress calculation by adding the found duplicates to the progress. Also states them in the details window.
  • Fixes a locking issue for the meta files (*.txt) which resulted in incomplete downloads.
  • Updates the details and settings view for a better understanding on how to use the application.

2016-11-20:

  • Fixes proper counting of downloaded files.
  • Fully implements the details window (context menus, etc.).

2016-11-18:

  • Fixes the initial automatic queue restore function.
  • Fixes the autodownload function.

2016-11-16:

  • Picture- and videopreview in the details window.
  • Allows the download of text, audio, quote, conversation, link type posts.
  • Download of text, audio, quote, conversation, link and .gif images are based on each blog instead of a global setting and can be turned on/off in the details view. The settings in the settings window are used as template for newly added blogs.
  • Modified .tumblr index files get now always saved upon application exit regardless of the crawlers state. Previously if the application was closed during an active crawl, the index wasn't updated.
  • Inlined the WAF code under lib for easier project setup for newcomers that want to contribute code.
  • bugfixes, UI and memory enhancements.

2016-10-15:

  • Bandwidth throttling.
  • Connection timeout settings.
  • auto queue and start download function.
  • save states of the UI (column size and order).
  • download of hidden blogs.
  • fix proper saving of the ratings and tags.

2016-06-11:

  • Added German translation.

2016-06-10:

  • Support for tumblr.com hosted videos. Check the settings window to enable video download (default: off).

2016-06-08:

  • Tag crawling now properly working. Also it's case-insensitive now.
  • Fixed crash upon blogs with zero-image count in the queue list (e.g. blog is offline, or tag search didn't evaluate any images).
  • Fixed randomly occurring crash in the clipboard monitor.
  • Changed icons (requested by the TumblOne creator).

2016-04-12:

  • Now with progress output in the Queue tab (during url crawling for imageurls -- the number of posts evaluated; during downloading -- the current image url).
  • Added missing resume button in the taskbar control.

2016-04-11:

  • Support for urls starting with https:
  • Fixes application crashed upon pressing the stop-button due to improper exception handling
  • Now saves the index file at every time. Previously the application would exit if the crawling processes was still active without properly waiting them to finish and save its state. Now there is a grace period for the tasks to finish. Same was true if the crawl was paused and then exited.

Download:

Comments

Slow (not verified)
Sun, 29/01/2017 - 17:10

I have discovered that if you try to use the "swipe" type of function with your mouse in the "comments" field of the main window pane it will cause a tumblthree crash and an error report is sent to Microsoft.

I was attempting to delete part of a comment by swiping over the part I wanted to delete and as soon as I moved the mouse the program crashed. I did this twice to verify that is what caused the crash.

Any kind of copy/paste operation using the mouse also causes a crash of thmblthree.

Anonymouse (not verified)
Wed, 01/02/2017 - 23:46

Is it possible to see the roadmap or any plans for the development the functionality of the program? I would like to see that is planned to develop or implement any feature in the program. I was also interested it is planned to make the file names more informative and useful for users. I would like to see in the file names instead tumblr_mzexuzeWz01rzhnu9o1_500.jpg such names like as id_ description_and_tags_to_the post_and_number_of_likes_and_repost.jpg or something like that.

zab
Thu, 02/02/2017 - 08:40

I can write a list of possible features (and bugs) together at some point. It's probably beneficial if someone wants to participate at some point he could directly find a task to work on. I was thinking of doing this as well ..

Taranchuk (not verified)
Fri, 03/02/2017 - 01:30

I rename tumblr files, adding to filenames caption and tags, for a long time by using metadata, just convert it, using a macro with huge number of regular expressions, to a file list for further renaming files. In my case, schema converting filenames looks like that:

base file name (tumblr_* without extension) - if this post is reblog, then there adding name blog where made reblog - text of photo caption and tags.extension.

Thus, I have file names looks as follows:

tumblr_momf2oK9UH1rdpzlpo1_1280 pinkfineart - tumblr reblogged blog - Elle Richie from Only Secretaries httppinkfineartcomonly-secretarieselle-richie-shift-dress67792 Elle Richie.jpg

The only drawback of this approach is that the metadata contains information about only one image of the photoset, so that the resulting file list becomes incomplete, so it can not be used to rename all the images from the photoset, it renaming only one image from the photoset.

Also, often the aggregate text with caption and tags and other information is too big to renaming because of the maxpath limit in Windows, thus need remove all after 220 characters in lines, except extension.

I can send you per mail macro in Notepad++, which transforms the metadata in the file-list, to learn and use.

anonymous (not verified)
Sat, 11/02/2017 - 15:49

Hello zab.

I tried latest 1.0.4.16 but it didn't worked at all for me, so I had to switch back to previous (.15) version. As I run crawling process it kills my Internet with a few dozens of TCP connections to Yahoo server, and unless I close the program I could neither open any site nor even ping them. Text "downloading file ...." is displayed in queue, but no files are being saved in blog folder. I've tried to limit number of connections and bandwidth settings without any effect. Deleting settings folder in userfolder didn't solved the problem.

zab
Sat, 11/02/2017 - 16:55

Thanks for letting me know so quick.

That's why I don't code if I don't have time. It's certain to happen a big clusterfuck :(

I've fixed it again, I hope it works now.

Thanks again! And sorry for all these bullshit releases .. :/

Rachel (not verified)
Tue, 14/02/2017 - 19:45

When I started the program it auto loaded a blog but received the error message 'could not download blog (insert name).' and when I tried to delete it, it says 'could not remove blog (insert name). What's up?

zab
Tue, 14/02/2017 - 20:37

Probably something with the index file. Either there was no space to store it, it got corrupted or maybe you have no permissions in your download folder?

In your download location there should be a folder called Index. In there should be the file with the problems. Maybe you move it out of the folder and try again. It's hard to tell from here.

You could also send me the file per email and ill check if it loads for me.

Slow (not verified)
Sun, 19/02/2017 - 03:08

I am totally confused by this statement:
"Decouples the scan from the bandwidth throttling and connection settings. If you've used an older version before, make sure the Limit Bandwidth (kb/s) value in the settings is below 2 million."

The default number that is in the box is: 2147483647 if I add the commas to assist me in reading the number in traditional American numeric's that number is 2,147,483,647 which is greater than 2 billion.

To add to the confusiuon you use the term (kb/s) which stands for kilibits/per second, but if you hover over the box where you put the number in, the help displays says kbytes/s. In either case, the number is in the billions which is 100 times greater.

The common syntax for kilobytes is kB and for kilobits is kb.

Assuming your usage of kb really is kilobits and not what the help says, it is stilll 100 times greater than what the comment reads.

My bandwith is actually 119mega bits per second.

Because the number is so huge, it appears what is in the box is really the bits per second, because if it was the number of kilobits, then that number actually has three more zero's after it to reflect what the box is named.

For instance if the box is really kb or k/bits then a value of 1 in the box represents 1,000 or one thousand bits per second since 1 kilo bits is one thousand bits.

If I use what the box says the number is expressed in then to tell it the bandwidth is less than 2 million bits per second the number would be 1999kb. Expressing that in bits it comes out to be 1,999,000 since the box is asking for k/bits and not raw bits/s. what is in the bok is actually 2,147,483,647,000 or 2.1 tera bits per second. Thats trillion!

What should really be in that box, and can the default number that comes with the code please be within the range of the statement in the release announcement.

I have never added any number to that box since I have been running tumblrthree, so that number is what comes with the code.

I am leaving the default number in the box and pray release 18 will not mess up my present assortment of images.

Thanks for a great tool and sorry I am so confused by this one box.

zab
Sun, 19/02/2017 - 04:16

No commas, but lower the value in the box to below 2000000.

Otherwise, depending on your parallel connections setting the download might stall as the value might turn negative. the 2147483647000 is the maximum value you can store in an integer. Now, if you add something, it will turn negative. The bandwidth throttler takes bytes as argument, so later in the code i multiply it with 1024. Now the value should be negative and not work. But at the same time it gets divided by the parallel connection balue which always masked this issue, since it starts with 25 an luckily bring the value back into the positive range. If you set it to 1 you should not be able to download at all. You can give it a try.

It you set it below 2000000 it just can never happen, regardless of the parallel connectios/parallel blogs setting as its always below the maximum value.

I really never should have started with the upper boundary of the integer value ..

Slow (not verified)
Tue, 28/02/2017 - 01:45

Thanks for the explanation. I used 1999 and it ran very well. In fact much better than it had been running before. I'm still not really clear if the box is kbits or kbytes, so figured I would play it safe and use 1999 figuring it might be k/bytes.

zab
Tue, 28/02/2017 - 17:22

kbytes.

But the downloads are so quick for (small) images that it's more a rough estimate because the throttling is based on seconds. You can open your task manager and check the Network throughput there.

Preston (not verified)
Sun, 26/02/2017 - 08:42

Hi there. I'm having a lot of blogs appear offline on Tumblethree, when I actually go to the tumbler site, that blog s very much still active and up. Do you know what might be going on.

Zabil Hyun (not verified)
Thu, 09/03/2017 - 04:15

i has downloaded all the post image and videos its more than 30.000 items, the pic name like tumblr_oHjg6xxxxxx.jpg" its okay im not realy neeed to rename pic but the videos name like "tumblr_tumblr_o29xxxxx.mp4" i need to rename it
when i'm watch on the tumblr blog, there is tags and description wich thats the name of videos.. now im confused how to rename the video name.. wich the video in my folder is 800~ item, the one way that i know is open the blog in browser and search videos one by one and see the post and copy paste the description under video post and rename it on files in my folder, but damn, its more than 800 videos.. can you help me to slove this problem? or maybe you have a way to download the description file (maybe)?
*sorry my bad english

Anonymouse (not verified)
Thu, 09/03/2017 - 07:47

II've been doing this for a long time, I rename the files using metadata that contain filenames, description and tags. Just set the download mode for the video metadata, and then use regular expressions to modify the text of the metadata so that it becomes a file-list. Then I just use the Total Commander to rename the files by this file list. If you do not know what regular expressions are, write me about it, I will write a list of regular expressions so you can use them for converting metadata to a file list.

Jun Manne (not verified)
Sat, 08/04/2017 - 23:02

This is an excellent application for backing up a blog but if the blog requires a login it cannot connect to it. Is there a way to backup that kind of blog where a login in is required?

Drobb (not verified)
Thu, 20/04/2017 - 10:09

Im wondering, hoping that there is a way you can implement a way to have the downloads be saved and stored either by making sub folders based on month/year or just renaming of the files or something to actually be able to store them chronologically so that it can actually be used as a sort of backup. It would be nice to be able to look back and see what posts were done when and etc. Love the work you have done. Been using this since TumblrTwo and all the various increments and updates. Great job.

zab
Fri, 21/04/2017 - 18:54

it's easy to set the creation time after the download. We are actually grabbing the post time too, so we could at least set the file date to the post date without any hassle. Then you could sort by date in the File Explorer?

Drobb (not verified)
Sat, 22/04/2017 - 02:56

could you explain how this is done? all i see are filenames tumblr_xxxxxxxx.jpg/png etc or tumblr_inline_xxxxx.xxx

Anon (not verified)
Tue, 02/05/2017 - 19:31

Hella! Been using TumblOne for a while and just noticed there's an improved version. May donate something. However with this one the program "evaluates" for the post with the specifications requested (text, image...) but never actually starts downloading anything. It actually did once when I removed the tag (xxx-yyy) I've been using but didn't go all the way through the blog. Any ideas?

zab
Tue, 02/05/2017 - 22:23

Sorry, it's too vague for me to give any tips. Try the latest beta as it fixes a downloader stall. I haven't really tested the tags implementation since the code rewrite, maybe it's broken right now. You could try the v1.0.4.31 release and see if thats any better.

Anon (not verified)
Tue, 16/05/2017 - 06:33

I'd like to download the entire contents of a particular tag on a blog, but when I restrict it by tag, it only downloads images, no matter what post types I tell it to grab. Am I doing something wrong, and if not, could you add a feature that allows downloading, say text posts by tag?

zab
Tue, 16/05/2017 - 07:24

Fixed. I've introduced that bug during my code refactoring/cleanup.

Should be back working now. Thanks for reporting!

anonymous (not verified)
Sat, 27/05/2017 - 11:38

Hello, I'm having issues with "Check blog availability on startup". If the option is enabled, TumblThree became unresponsible or shortly after being launched. I have about 30 blogs in database. Does it take too much time to check them all?

zab
Sat, 27/05/2017 - 20:09

It's obviously not related to the amount of blogs.

I don't know. Check all of your blogs, must be one blog/file specifically. You could move each of them out of the \Index folder in your download location and check if the application still hangs. Then send me that broken file, then I can take a look into it.

anonymous (not verified)
Sun, 11/06/2017 - 16:49

Sorry for the late reply.

I've tried to conduct a few experiments:

  1. At first I suspected that the reason is me messing up with downloaded files, as I replaced the unneeded downloaded images with zero-size files to free up some free space (it was webcomic blog with a lot of annoying page teasers and such). So I deleted the blog folder and removed the blog from bloglist, but the problem still remained.
  2. Then I moved all blog files to another folder, purged blog index and started to move files back, re-add blogs one by one and restart program every time a new blog was added. The problem showed up again after 10+ blogs.
  3. Finally I moved all blog files to another folder, purged block index (so the database & filebase was empty). Prior to any following actions I've launched ping 8.8.8.8 -t. Then I launched the program and just copied the list of blog URLs into the clipboard. Even adding a list of blogs triggered the problem, and it took about 7 minutes just to add 30 blogs (without downloading files), but this time there were a few ping timeout waves during that process.

So it seems that the root of the problem is network congestion.

zab
Sun, 18/06/2017 - 12:59

I've just tested it with 135 blogs, turned on the Online Check during Startup-option in the general setting. No unresponsiveness.

It acquires access to the the api for the online checking, that's why it's off per default now. Thus i basically reduces the possible downloadable file detection throughput if you immediately start a crawl processes after application startup.

anonymous (not verified)
Sat, 27/05/2017 - 12:39

Hello.
I've spotted some strangeness in /Index folder.
For some blogs there're two files: blogname.tumblr and blogname_files.tumblr.
For other blogs there're three files:blogname.tumblr, blogname_files.tumblr and blogname_files_files.tumblr.

anonymous (not verified)
Sun, 11/06/2017 - 17:26
  • Is it possible to show the state of blogs' checkboxes (selected/partially selected/clear) on "Details" tab when multiple blogs are selected? Now all checkboxes are shown as clear and it's confusing.
  • If preview images on "Details" screen had different dimensions (especially height), the blog settings are rapidly jumping up and down with each new image being displayed. Maybe it make sense to align preview image to the bottom if it's larger than height of the "Details" tab?

Anya (not verified)
Wed, 14/06/2017 - 08:09

So i downloader TumblOne yesterday, and i didn't know where did they save my files, it kept on directing me to the C:/Documents one but the files are not there, i tried using another Tumblr blog and the right one [which is the blog] file popped up, and now i couldn't find the first tumblr blog that i have crawled, and it cost me a lot of gigabytes, idk where it is please heeelp

zab
Wed, 14/06/2017 - 21:02

TumblOne stores all its files in the Subfolder Blogs relative to the .exe file. So if you downloaded TumblOne.exe into C:\Documents\TumblOne.exe, there should be a folder C:\Documents\Blogs\ with your downloaded files in it.

By the way, I'm not the author of TumblOne. If you want any further support, please ask your questions at the appropriate sites.

anonymous (not verified)
Sun, 18/06/2017 - 23:27

As it turns out you can download the higher resolution of an image from tumblr by downloading the raw untouched image. You can do this by replacing "1280" or whatever number it is to "raw". Not every image will be larger from this, but I have seen a few that will get you the original image that can be quite larger than its 1280 resized verison.

I would like to see tumblthree get updated with the option to download the raw images soon as I now realize that I will absolutely need to redownload all of the tumblr blogs I got.

anonymous (not verified)
Thu, 29/06/2017 - 00:19

Thanks for adding the raw image support, its been very helpful as I been going through my list of tumblrs.

problem is that I am finding quite a lot of images that don't work with raws as it will give you a 404 error. I was pointed out that removing "68" from the image url at the beggining will fix this issue.

like "https://68.media.tumblr.com/" turn it into "https://media.tumblr.com/" with the direct image url to the raw image will fix this.

also have you heard about tumblr turning all nsfw blogs private? You will be required to log in to see them sometime early next month. Will we be able to use tumblthree on them when that happens?

zab
Thu, 29/06/2017 - 07:49

No, I haven't heard about the mandatory turning into private blogs. Do you have any links with more information?
I just briefly searched the web, but could only find a new "safe mode" filter which seems to be already deployed but luckily TumblThree still seems to work fine.

If they'll all require a login, they will probably not work anymore. But we'll have to see and everything is fixable somehow.

Rakesh (not verified)
Wed, 21/06/2017 - 13:19

app is not responding and getting closed, followed below instructions

Go to Settings, click the Authenticate button. Logon to tumblr using an account. The window/browser should automatically close after the login indicating a successful authentication. TumblThree will use the Internet Explorer cookies for authentication.
Add the blog url including the liked/by string in the url (e.g. https://www.tumblr.com/liked/by/wallpaperfx/).

zab
Wed, 21/06/2017 - 19:19

Fixed. I didn't check what happens if the grabbed post information does not contain a valid date. In the newer release I set the files modified date to the post date, but the TumblrLikedBy grabber doesn't support this.

anonymous (not verified)
Sat, 24/06/2017 - 15:05
  1. Is it possible to add option for sorting original and reblogged post, so rebolgged ones could be automatically downloaded in subfolder? That way path to original files is ./Blogs/BlogName and for reblogs it coul be ./Blogs/BlogName/reblogs.
  2. If process of blog crowling didn't end up successfully (application crash, network is down etc), sometimes after re-starting it file counter in "Downloaded files" tab begin to count downloaded files as not-downloaded. For example, if blog had 100 downloadable posts and crawling process was interrupted on 98th, re-staring and successfully completing crawling process will result in 100 actually downloaded files, but downloaded files counter will say there's 198 files. If on second try it will be interrupted and restarted again, it will show 200+ files in the tab etc.

Stan (not verified)
Tue, 04/07/2017 - 16:59

Thanks for the great program. I've downloaded the latest version and simply get a crash every time I attempt to add a blog. Any ideas?
Win10 64bit. .Net framework is installed.

zab
Tue, 04/07/2017 - 19:12

You need to login to download anything in the v1.0.7 release as I've written it there. That includes adding a blog as it checks its online status: Go to Settings -> Authenticate.

Sorry for the crash, I didn't check what happens when adding a blog without a valid login. I've only checked the actual download, which I properly handle.

zab
Wed, 05/07/2017 - 01:36

I'm sure you've downloaded the source code and not any release file because there is no TumblThree-master.zip release.

Please read the page properly before posting anything here. The download is here.

zab
Thu, 06/07/2017 - 07:54

Thanks for the info.

I've actually tried that already but it didn't seem to work always either. I don't know. It kinda sucks try out 6 different urls in order to download anything.

Pages