Saturday, November 6, 2010

How to download and backup your GTalk/GMail chat logs

Mostly taken from: http://martinml.com/en/how-to-download-and-backup-your-gtalk-gmail-chat-logs/ but includes my developed script

Why?

I use GTalk a lot. I have stored thousands and thousands of conversations for about 5 years. And it's all there, in GMail, without being able to backup it locally, unlike normal e-mails.
Why would I want to backup my chats? Because I don't want to lose them, and because GMail is awfully slow when having more than a particular number of e-mails, and I think that includes chats. Also they show up in every search when I try to find an e-mail in particular.

Invalid solutions

  • Labeling the chats and downloading them via IMAP: it looks like it worked (by accident?) in the early times of IMAP in GMail. Doesn't work anymore.
  • Forwarding the chats to yourself: you have to do this one by one. It's not a very good method.
  • Creationg a rule that forward the chats to yourself: it doesn't work with already existent chats, and I'm not sure it'd work with the new ones.
  • Using a script to parse the Gmail website: if you have thousands and thousands of chats, like me, you will get blocked from your own account in a few hours. And it is a violation of the TOS, so you could lose your account permanently.

The solution

There is a function called Offline Gmail which allows to use Google Gears to download your e-mail to your browser so you can use while offline or when your connection is not reliable. For that Go to Google Gears and check your installation status, if it says Installed, proceed below. If not, install it and restart your browser. Google Chrome has it installed by default.
Now, You can choose to let Gmail decide which e-mails download, or you can specify a label. But... What if we marked our chats with a label...
So first search within All Mail as follows:

Uploaded by CandorZ

So the result would be as follows:
 
Uploaded by CandorZ

Now Select All the Search Results and Apply a new Label as shown, mine is LOGS:
 
Uploaded by CandorZ

Your Result should be something like below:
Uploaded by CandorZ

Now Go to Settings -> Offline

Uploaded by CandorZ
  
Enable Offline and Expand Download Options:


Uploaded by CandorZ

Select Options as you may please (I'd advice only to select LOGS)

Uploaded by CandorZ

Click "Save Changes". It would show you some progress as below:

 


After finishing you will have to locate the offline file. The browser I used was Chrome and the path you can see below, mine was: D:\Documents and Settings\USER\Local Settings\Application Data\Google\Chrome\User Data\Default\Plugin Data\Google Gears\mail.google.com\http_80. Highlighted is the file (It's size would be large as compared to the others)

Uploaded by CandorZ

Copy the Database to the location/folder where you have the two files, namely extract.exe and sql4.vbs. You can get the files from here 1 & 2. Rename your database to candorz (It will show you an error otherwise "File Not Found")

Uploaded by CandorZ
Now Run the sql4.vbs, just double click it!!! A subfolder "msgs" would be created which will contain all your LOGS. Enjoy!!!

Uploaded by CandorZ


21 comments:

  1. Wow! Thanks a bunch for that. Had I known this before... It would have been so much easier to bakcup my chat history!

    ReplyDelete
  2. Help. I get the error message Microsoft VBScript runtime error when I try to run Sql4. am I doing something wrong?

    ReplyDelete
  3. Help. I get the error message Microsoft VBScript runtime error when I try to run Sql4. am I doing something wrong?

    ReplyDelete
  4. it comes when you dont rename the db as candorz

    ReplyDelete
  5. Copy the Database to the location/folder where you have the two files..

    i didn't get this step. where to copt database. plz tell me..

    ReplyDelete
  6. This is amazing. Thank you so much!

    ReplyDelete
  7. This is excellent. Any way to automatically save the date/time of the chat to each .htm file?

    ReplyDelete
  8. Thanks for the awesome script. I wanted to transfer my chat history to another google account, and stumbled upon your guide. With a few modifications, I was able to create .eml files that preserve the time/date/sender and so on. The .eml files can then be imported into a mail client such as Windows Live Mail, from which they can be uploaded to google via IMAP. Of course, the .eml files might also be a good way to archive the chats.

    I've put the modified script up on my blog at http://bits.karlrees.com/2011/01/google-talk-chat-history-export.html

    ReplyDelete
  9. The only **problem** is that if your chat is really long, it will clip it and not download the whole thing >.<
    [and it will put some text at the end saying "this message was clipped. click to download full message" or something like that]

    ReplyDelete
  10. everything is happening fine. the html files also got created. bt they r empty. contain nothing. no script no data. :( what to do.

    ReplyDelete
  11. Dude.. this is damn smart.. thanks so much.. i will just wait for a while before i post this on my facebook.. so that all people can use this. thanks.

    ReplyDelete
  12. Although what your script does is commendable and i have no reasons to complaint, but just to rest my inquisitiveness, is the script not supposed to save the date of the chat? or m i doing something wrong. n secondly, there are a few junk characters on the left hand side of each line. Â. like this. why are they coming? are they supposed to be there, or again, m i doing something wrong?

    ReplyDelete
  13. I was testing Karl's EML output approach earlier today, and I've come across a limitation in the SQLite dataset.

    Looking at the output HTML right now, I see several chats that are truncated with the "[Message clipped] View entire message" footer. While it's not entirely consistent, any output HTML greater than 90kb has a strong chance of being truncated in this way.

    It appears to be a limit on the SQLite dataset. If I run the following query against the SQLite offline database I get several 38 hits and I have 42 HTML files 90kb and higher.

    select * from MessagesFT_content where c0Subject like 'Chat with%' and c1Body like "%View Entire Message%";

    ReplyDelete
  14. @J Ruske, do you know of a way to solve that error? I mean, is there a way to make offline gmail download "clipped" messages fully?

    ReplyDelete
  15. this worked wonderfully for me. one thing i am wondering, however, is why the messages don't have date and time stamps.

    ReplyDelete
  16. Thanks a lot! It worked perfectly.

    ReplyDelete
  17. i want to download a particular labeled chat to word doc..label contains almst 500 mails and some of them veryy long almst 1500 lines ..
    how do i go about it ?

    ReplyDelete
  18. About 75% of the html files it created are empty. Any reason for that? Am I doing something wrong?

    ReplyDelete
  19. Google Gears support is gone. Solution doesn't work.

    ReplyDelete
  20. Hey folks,

    I've come up with a simple way of doing this that still works. Hope it helps everyone, this problem has been bothering me for months!

    Check it out here:http://freshhorse.wordpress.com/2011/07/29/leaving-gmail-and-bringing-your-chats-with-you/

    x7o

    ReplyDelete
  21. Hey x7o2e7o, I got this:

    http://freshhorse.wordpress.com/ is marked private by its owner. If you were invited to view this site, please log in below. Read more about privacy settings.

    ReplyDelete