Getting here took about 2 weeks, and it was a nightmare. It all started because I wanted to have certain things on all my blogs, but how do you do that when every blog is separate, with its own template and login details? I really didn’t feel like changing it on 15 blogs. So I slowly started working on a Blogger uploader.
It was day 1, February 26. First I took a template and tweaked it a bit. Now it was flawless :-). It came as no real surprise that the template couldn’t be uploaded in the new Blogger interface, but worked fine in the old one. That was February 26. In the end I decided the blog really would live on Blogger. I basically had only 3 options: Blogger, WordPress, Posterous. WP somehow didn’t suit me, Posterous let me down, and so did Blogger, but it was the lesser evil. That same day I also tried putting it on my own hosting. I set up free hosting at Endora, but I couldn’t get the CNAME records in DNS set up, and on top of that I would have had to show an ad in the footer. I also tried various other free hosts, but each one had some flaw. I tried converting the template to the WP format, but got stuck on the fact that I don’t understand PHP.
The next day, February 27, I started writing the program itself. First the parser classes, which I planned to release freely on the internet. That’s why I gave all the items English descriptions and variable names, and the properties were in English too. Then I tried posting to Blogger. But alas, sometimes it returned just an exception “Execution resulted in a redirect from http://google.cz and other times “method not allowed”.
I have no idea what it was. I tried solving it in various ways, downloading gdata for Java, Python, JavaScript. With Python I found out that I just can’t handle that language, JS would have been clumsy, and Java – that is the greatest evil. I installed maybe 7 IDEs, all of them of course painfully slow and mostly written in Java. But in none of them did I manage to add those Java files to a project!!
So in the end I gave up on Java and swore: Java, never more! I even tried decompiling the gdata .NET library, but of course it threw a million errors at me. I fixed them as best I could, but then I ran into an error I couldn’t fix – namely, a class was compared with the same class via an overloaded == operator, which in turn just called itself.
Finally I tried a sample straight from Google‘s website and, lo and behold, it worked. Lesson learned: don’t use code from random websites when there is official code available. It may not be up to date, and so on. It might also have been because I wasn’t entering the email as sunamoCZ@gmail.com but as sunamocz@gmail.com. After that, the exception “Execution resulted in a redirect from google.cz” made sense.
Of course, being as blind, nervous, dumb and prone to assuming things as I am, I ran into things like doing everything with “public”, and only when the program didn’t work and I debugged it did I find out it was supposed to be publish!! Today I also finished the WXR file parser. Of course full of bugs, but some sort of alpha version was done.
In the evening, at 8 o’clock, I already had a form that let me check which posts I wanted to upload: The screenshot on cl.ly expired long ago.. However, the form couldn’t do anything else yet. At 8:20 it dawned on me. I realized that I couldn’t upload this with WLWriter and that I would have to handle the upload to Picasa myself. Back on February 26 or February 25 I had looked into plugins for WLWriter, but found out that the kind of plugin I needed couldn’t be made – you can upload at most one post, with the option of sending it to several servers. Now it occurs to me that it might have been possible, but back then I concluded it really wasn’t.
The next day, February 28, I started programming a bit more. First I wrote a completely insane loop to calculate the thumbnail size, gave it 4 different inputs, and one of them returned a wrong value. Luckily, a good buddy of mine showed me how to get rid of the loop, and it even returned exact results every time!
Specifically, my method with the loops looked something like this: https://pastebin.com/E3xiSb65 . That day I still didn’t know what to do with file types other than images, for example .pdf, .doc, .mp3, etc. Then I didn’t know how to save images into the folder so that I could tell whether the image on disk was the one I needed to upload. I had these 3 options:
- Save files as, for example, 1.jpg and keep a dictionary with their real names
- Append something like “file_size_45678” to file names, and if a file already contained “_size_”, replace it..
- Save into subfolders “45678/file” (a bit more implementation)
In the end, option 1 won. Then that day I was figuring out how to get rid of the “Picasa HTML entities” the API returned. Those weird characters appeared when the image was on the ggpht.com server, so I uploaded every accented letter as an image to Picasa one by one and experimented. In the end I came up with 3 different dictionaries and that’s where I stopped – it behaved differently every time.
Next I wanted to do it through the IE-based WebBrowser control, but I wasn’t too keen on that, because WB needs MTA and I don’t like it, since with it the other half of my program doesn’t work. And above all, I don’t like IE :-). I also tried downloading it in all browsers (Opera, Firefox, Chrome, IE) and they all worked, only the .NET classes didn’t :-(. So I tried Awesomium.NET. The first problem was that I couldn’t compile the C++ wrapper around Chromium.
I didn’t take a single day off, and the next day came, February 29. Today I tried to download those images to disk using .NET classes. However, it didn’t work at all; I tried various headers, etc., but I always just ended up with an error. Fine, so I did it the way I had done it with the Cammins. I generated an HTML file, opened it in FF and downloaded it. But alas, it always downloaded just an HTML file that only contained a link to the downloadable image.
On top of that, it destroyed the diacritics, so “i ลพe” turned into ” e”. Today I also wrote a lot of code, but my bad luck wouldn’t have shown itself if something hadn’t gone wrong – my PC apparently overheated, shut down, and after starting VC# again, AutoRestore didn’t work ๐ . Until now I hadn’t thought about publishing the program so that everyone could use it. Today I also messed around with WordPress – I was trying to import my Blogger blogs into WP and then export them to get a WXR file, but WordPress is awful junk – for example, WP can’t remove a category from a post – that is outrageous. I tried it every possible way, but it just didn’t work. Finally, late at night just before midnight, I decided that the program would check for duplicates on the web using a file it keeps on its own disk.
I stayed up all night into March 1. Within 1 hour I had it programmed; it was supposed to work something like this: if already uploaded posts are found, they are skipped and saved..after the current loop finishes, a dialog shows the possible duplicates found..the user picks what they want to upload anyway. (Hm, it just occurred to me again that I should have made it so that clicking once checks the item and clicking twice opens the blog post it collides with, but I would have to extend it a bit for that, because right now it only stores the source blog and the post title)..
when they check something and click “Upload“, it gets uploaded… At half past six in the morning I tried to import at least the Posterous blogs into WP. But WP couldn’t really do even that. So I downloaded 2 Posterous .NET APIs. I went on working on CopyBlogForm. That was a form where you would be able to convert blogs in any combination from Blogger/Posterous to Blogger/Posterous. Then I tried uploading to Posterous through the API. That library had just one method for this, and it required a PosterousNewPost as a parameter. Once again it showed that I can’t read and how scatterbrained and unfocused I am, because I “saw” that it needed a PosterousPost, which couldn’t be created in any way – it wasn’t public, the class created it internally. Through the API console at posterous.com/api I could add posts just fine, so I downloaded Wireshark and tried to copy what it did.
I tried it with WebRequest, HttpWebRequest and WebClient with exactly the headers Posterous sent – that still worked. But as soon as I didn’t send the cookie header, whose value contained about 20 differently generated strings in some encryption, encoding or charset unknown to me (I tried every encoding in .NET, and base64 as the encoding)… Because the .NET classes for working with XML are terribly clumsy, I looked for my own classes that would make working with it easier. I didn’t find anything that was exactly what I was looking for, so I gave up on that and did it with XmlGenerator at the level of plain strings. On top of that, about that Posterous API console – it wouldn’t tell me my API token. On another account it did, but this one was created specifically with a different password just for testing the API, and here it somehow didn’t work. But I didn’t know that yet back then. And that account was strange altogether. The very first time I want to load 6 short posts from Posterous (there aren’t any more) via the API, and supposedly it’s 403 Forbidden (Rate Limit Exceeded)..
It was past noon on day 6, March 2. I was agonizing over the fact that I still couldn’t figure out why uploading to Posterous didn’t work. I blamed it on the fact that the API I had was v1 while the current one was v2 – so that, for example, the DLL generated a wrong auth token. Maybe v2 only supports OAuth/XAuth. But I can’t say, because I never found anything about it anywhere. Stupid Posterous. Honestly, the only resource on the entire internet was that API console. As a very last resort, I tried requesting the API token.
So I did, and they promised they would get back to me soon. The whole time it seemed nonsensical to me, because if it was the token for my account, anyone could work with it (even if I encrypted it with an asymmetric key, anyone could sniff it from the headers and the connection as a whole, so the only option would be encrypting it with SSL, but I don’t know whether the .NET classes can do that.) In the end I threw in the towel and declared that it would import only to Blogger. That’s also why my program is called SunamoBloggerBatchUploader.
After that I was in Poland and in Cieszyn shopping and running errands with my parents, and I bought a lamp for myself. I’ll write a separate article about that. At 4 p.m. I was still dealing with the authorization. I read a few things about it online, but found no solution. I only found out that the auth_token sent by the API console was different from the auth_token my library sent, and also that it was not in Base64, as I had read everywhere on the internet and above all in that .NET DLL. Then it occurred to me to look at the source code of Writerous. But it used the same library I did, and so it didn’t work either. After that I concluded that it doesn’t work for anyone and I gave up on it.
By March 3 my program could already log in to Posterous and Blogger and save the credentials in the registry if the user wanted. It could also download blogs to disk – both the WXR/XML files and all the images. Then it can work with those WXR files – search in them, create custom ones from existing ones .. then in the CopyBlog form it shows what the program is currently doing, and it has a progress bar.
It keeps track of uploaded posts, and when it finds one that has already been uploaded, it skips it and at the end asks what to do with each such post. Uploading selectively – meaning the user can pick just the posts they want to upload and doesn’t have to do it in my manager or, God forbid, by hand. I didn’t work only on the BloggerUploader but on other things too; for example, in ParsovacCammin I added support for parsing Cammins downloaded from House-Mixes. And also a Converter of solutions and projects from other versions to version 2005. That one, for instance, had the problem that it refused to compile LINQ: The screenshot on cl.ly expired long ago... so after many attempts I rewrote it in plain C# :). The first converted project came out fine: The screenshot on cl.ly expired long ago..
I tried converting a few of my projects; I haven’t yet tested whether they work – I know it only threw warnings: In the evening I worked on CreateCustomWRXForm. That’s a form where the user can choose what they want in a new WXR file, then just enter a name and it gets created. I also found out that I need to add s1600/ before the file name, which I didn’t understand, because as far as I could tell, it hadn’t been quite so necessary yesterday. So I fixed that and started uploading posts through the API. But alas, I soon found out that I can post only 50 posts to Blogger and then only after 12 hours another 50. Once again it reported the all-explaining error “(503) Server Unavailable” or “(400) Bad request”. I tried setting a delay of up to 6 minutes, but it didn’t make much sense.
March 4 brought another change: upload to Picasa (I hope there’s no such limit there as on Blogger), generate the XML (also my program) -> import into Blogger via the web. Then I was dealing with the fact that the XML imported into Blogger fine, but not a single post got imported. On top of that I was dealing with internet problems, because some applications were downloading at only 3 KB/s and the upload was about 10 KB/s instead of the usual 20 KB/s. Moreover, instead of images, only black warning triangles were showing, and to top it all off they didn’t respect the column width, so I’ll have to fix that too, because it still isn’t fixed as of today (March 8): The screenshot on cl.ly expired long ago.. Likewise, to this day my program can’t export from Blogger (that’s because the first time it returned verbose HTML, but after that normal XML, which was easier to make sense of – hopefully – I haven’t studied it much yet. Of course, even here, when I tried uploading to Picasa, I didn’t avoid exceptions: “Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.” So I told myself, “if it does that one more time, I’ll wrap it in a try-catch, put Thread.Sleep(1000 * 60 * 2) in the catch followed by the same call..if it fails again, the program will just stop with the exception”. And it did, so I changed the code, and since then I haven’t run into that exception :).
March 5 – Today I learned that the source code of those web utilities for converting between various blog formats is on Google Code. Unfortunately it was in Python, and I really can’t handle that, so I tried to somehow convert it to C#, and when that failed, at least compile it as a .NET assembly. In the end I decided that from today on, Python joins Java on the list of languages I will steer well clear of. Finally I decided to try to find out what was wrong with my WXR export that made the web utility refuse it.
March 6 – I discovered Picasa’s limit: at most 1000 photos per album. Fine, so I wrote some code that, after reaching 1000 photos, would create a new album with a number 1 higher, so it would look like sunamo, sunamo1, sunamo2, etc. However, I made a mistake again: I did create the new album, but didn’t store it in the program, so it created the album again for every next photo! Luckily it only managed to put in 9 photos before I noticed and fixed the code. The Blogger API also refused to return the list of posts, which was strange, because when I put that address into Chrome, the result showed up just fine. And so far I have transferred 3 Blogger blogs this way, so you can’t say the .NET API didn’t work at all. Today I also looked into why some tags, especially <br>, overflow the column width. I tried various settings from width, max-width, overflow and others, until I finally tried the much-despised word-wrap. And that was exactly what I needed. It did break words in the middle, but at least it worked :).
March 7 – Today I finally put everything on that blog. And then I found out that .. everything was wrong :'(.
1) I noticed that from a certain point in 365, the photos were inexplicably shifted by 1. Also, some posts had a hard-coded black text color, so I couldn’t see anything in them. So I asked a buddy which tags I should search for. He told me to find out the classes and IDs of those blog posts. Well, yeah, but that wasn’t possible, since the export and import had wiped all of that out, if there had been anything. So in the end I did it with what came to my mind – the p, div and span tags. And it seems that was completely sufficient. Today I also made a program for managing, searching and replacing in my blogs. Who could make sense of that jumble of letters otherwise.. This link died before ICQ did.. Thanks to that highlighting, I knew I wouldn’t replace anything I shouldn’t.
March 8 – Today I got fed up and made a program for what I had been doing all day yesterday – that is, exporting to Blogger XML. Today I also converted the shared classes and methods in swf to English. All the other classes and methods in that solution are still waiting for me. The biggest trouble was caused by the Blogger date – first I thought everything had to be padded to 2 digits. When that wasn’t it, I noticed that in the time I was using – instead of :. And then something else, but I can’t remember exactly what. Each time I had to go through various exports and compare them so it would be exactly the same.
March 9 – I also worked on other programs. Otherwise, I continued with the motivation to publish this on the internet, so I built a completely new interface from scratch, entirely in English, which could convert various exports to various imports both crosswise and one-way, could remove content using preprogrammed parameters, could search in titles and content, etc. It could already successfully parse the export with all my blog posts, 1448 in total ๐ However, I also found out that I shouldn’t have deleted the “Windows Live Writer” album from Picasa. That’s because on about half of the blogs from Blogger and Posterous, it left links to the original photos and images, so I had to edit a lot of blog posts by hand and fix those photos.
March 10 – The final day after maybe 15 days of all kinds of work. Today I only edited various posts and learned that I shouldn’t replace & with ” and ” globally, because it ended up with the browser showing texts like “& ” instead of “&&”. I also hid other unwanted posts, such as unfinished ones; in total I hid 15. Well, I expected there to be more. And I’ve had just about enough of it, so now I’m off to publish my program, or rather its English part.
Conclusion: I spent a lot of time on this, but it will still take a lot of work before everything I programmed becomes usable. What annoyed me most is that the images are still on Blogger and Posterous, so I can’t delete those blogs. But whatever, at least I’ll be able to put a note and a link saying the blog has moved there. And now I’m off to write more posts for this blog..