Scripter Tool: Talk File Parser

[Kaelan 25]

RGSS3 Talk File Parser
Author: Kaelan



Overview

One of the biggest challenges in a text-heavy RPG is keeping all of your text data in check. This becomes increasingly more complex when you need to use text in ways beyond the limitations imposed on you by VX Ace's engine, such as the length of description text.

It gets even worse if you try to translate your game - now you have to dig through the database and your events for every last piece of text you've ever typed that's visible to the player, and manually translate them to another language. Translating to multiple languages - for those capable of doing so - is an incredibly time-consuming task.

This script was made to alleviate some of that effort. Traditionally what you would do in a commercial game is have a dialog file that holds all of your game's text in a given language; the game itself only holds the position in the file where a given line of text is.

To change the language you then just make a copy of the file with all the text in the same position, but in a different language, and tell the game to use this other dialog file. This allows you to change the entire game's language at will just by swapping one data file, rather than tinkering around with your actual events inside the game, which is not only more time-consuming, but more prone to errors.

This script provides that ability in RPG Maker VX Ace, through the TalkParser class. You provide it a ".TLK" file containing various text strings, and it will read the contents of the file for you, and be able to grab any string inside it and return it to you in RGSS3's environment, for use wherever you'd like inside your scripts.

Features

• This is a scripter's tool. It's meant to be used by others who script to facilitate development on text-heavy games. If you can't script, you probably won't get much use out of this
• Correctly parses ".TLK" files. It's assumed you're using v3.0 of the format. Not guaranteed to work with older versions.
• Once you've actually built your dialog files, this is absolutely trivial to use. One method call to load the file, and another to pull whatever string you want out of it.
• The text isn't altered in any way: whatever you type in that file will be passed directly to your game. That means you can format the strings for use with any other scripts that modifies text drawing or improves RMVXA's text display, like Yanfly's Message System

Usage

Once you have a ".TLK" file somewhere on your computer (I recommend storing them inside your game's Data folder), you initialize the parser with the file name, then call getLine() to grab a string from the file.

If I wanted to load one giant text file for my entire game at initialization, for example, I would do something like this:

 

module DataManager

  class << self; alias load_normal_database_kdea load_normal_database; end
  def self.load_normal_database
	load_normal_database_kdea
	$data_text = KDEA::UTILITIES::TalkParser.new("Data\\dialog.TLK")
  end

end

Then, later on, somewhere during the game, you can pull out the strings and do as you like with them:

	@actor.description = $data_text.getLine(149581)
	@actor.name = $data_text.getLine(149720)

It takes a while to setup, but once you're developing your game like this, all you need to do is swap the ".TLK" file say, to one with the same exact text in each position, but in Japanese instead, and now your entire game is in Japanese. No event or database editing required.

As a bonus, since it's just a file you're loading, nothing prevents you from even allowing people to change the language of the game while it's running. Writing the UI code to support that is left as an exercise for the reader.

Script

 

#==============================================================================
#
# â–¼ Kaelan's D20 Engine Ace - Talk File Parser v1.00
#	Last Updated: 2012.05.22
#	Requires: ---
#
#==============================================================================

#==============================================================================
# â–¼ Updates
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
# 2011.05.22 - Started Script
#
#==============================================================================
# â–¼ Comments
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
#
# Used to parse the Bioware .TLK file format. Call TalkParser.new() and pass
# it the file name of a TLK file (I recommend storing them in the Data folder
# of your project. Example:
#
#   $data_text = KDEA::UTILITIES::TalkParser.new("Data\\dialog.TLK")
#
# After that, you can use the getLine method to pull strings out of the file,
# by providing their string IDs. i.e:
#
#   @myString = $data_text.getLine(149581)
#
# It's your responsibility to know what your string IDs actually are. You can
# use a TLK editor program to manually create and edit the contents of a TLK
# file to use with your game.
#
#==============================================================================

module KDEA
  module UTILITIES

#==============================================================================
# â–  TalkParser
#  Reads data out of .TLK files. Assumes files are version 3.0. Not guaranteed
#  to work with any other version.
#==============================================================================
class TalkParser
  attr_reader  :loaded
  attr_reader  :fileName
  attr_reader  :version
  attr_reader  :languageID
  attr_reader  :stringCount

  SHOW_DEBUG_INFO = false # Set to true to see debug text related to what the
						  # parser is doing every time you call it, such as
						  # the contents of the strings it reads.
  HEADER_SIZE = 20		# Don't change this
  STRING_DATA_SIZE = 40   # Don't change this

  #--------------------------------------------------------------------------
  # initialize
  #--------------------------------------------------------------------------
  def initialize(fName)
	@loaded = false
	@fileName = nil
	@version = nil
	@languageID = nil
	@stringCount = 0
	@entryOffset = 0
	openFile(fName)
  end
  #--------------------------------------------------------------------------
  # openFile
  #--------------------------------------------------------------------------
  def openFile(fName)
	@mem_buf = nil

	if not fName
	  print("[KTP] No file name specified!\n") if SHOW_DEBUG_INFO
	  return
	end

	@fileName = fName
	@mem_buf = File.open(@fileName,"rb")

	# Read file properties
	# These are always in the first 20 bytes of the file
	@mem_buf.seek(4, IO::SEEK_SET)
	@version = @mem_buf.read(4)
	@languageID  = @mem_buf.read(4).unpack("V")[0]
	@stringCount = @mem_buf.read(4).unpack("V")[0]
	@entryOffset = @mem_buf.read(4).unpack("V")[0]

	if SHOW_DEBUG_INFO
	  print("==============================\n")
	  print("KDEA TalkParser Init\n")
	  print("File: " + @fileName + "\n")
	  print("Version: " + @version.to_s + "\n")
	  print("Language: " + @languageID.to_s + "\n")
	  print("String Count: " + @stringCount.to_s + "\n")
	  print("Offset: " + @entryOffset.to_s + "\n")
	  print("==============================\n\n")
	end

	@loaded = true
  end
  #--------------------------------------------------------------------------
  # getLine
  #--------------------------------------------------------------------------
  def getLine(strRef)
	return nil unless @loaded
	return nil if strRef >= stringCount

	@mem_buf.seek(HEADER_SIZE + strRef*STRING_DATA_SIZE + 28, IO::SEEK_SET)

	# String Data
	offsetToString = @mem_buf.read(4).unpack("V")[0]
	stringSize = @mem_buf.read(4).unpack("V")[0]

	# String Contents
	@mem_buf.seek(@entryOffset + offsetToString, IO::SEEK_SET)
	strContents = @mem_buf.read(stringSize)

	if SHOW_DEBUG_INFO
	  print("-----------------------------------\n")
	  print("[KTP] Reading String #" + strRef.to_s + "\n")
	  print("[KTP] StringOffset: " + offsetToString.to_s + "\n")
	  print("[KTP] StringSize: " + stringSize.to_s + "\n")
	  print("[KTP] Contents:\n" + strContents + "\n")
	  print("-----------------------------------\n\n")
	end

	return strContents
  end

end # TalkParser
#==============================================================================
#
# â–¼ End of Classes
#
#==============================================================================

  end  # UTILITIES
end # KDEA

#==============================================================================
#
# â–¼ End of File
#
#==============================================================================

 

Notes

I'm using ".TLK" files specifically because they're very simple to read and documentation on it is easily available. It's a widely known format, primarily used in Bioware games.

• You can find documentation on the format here: http://static.daniweb.com/images/attachments/2/Bioware_Aurora_-_Talk_Table_Format.pdf
• You can use this program to manually create, view and edit ".TLK" files: http://nwvault.ign.c....Detail&id=1005
• You should create your own files for use in your game. If you need a reference to start from, you can find them in various NWN2 modules people have developed, such as this one: http://nwvault.ign.c...l.detail&id=280 (Look for the "dialog.TLK" file and open it up in the editor provided above)
• Obviously you can't use this in a commercial game. The parser itself is entirely my own code and you can use it however you like, but if you're making money using files in the ".TLK" format, you need to talk to Bioware.

This is a pretty simple and yet pretty specialized script. I don't expect most people will have any use for it. But I needed something like this for my game, so I figured I'd share. I just implemented it recently, so let me know if there's any bugs. I haven't implemented using the sound file entries that are optionally included in ".TLK" files, since my game won't have any voiced dialog. If you need that, let me know and I'll write an updated version of the parser that handles those as well.

Credits
- Bioware, for making the file format documentation available for public use

Edited February 21, 2013 by Kaelan

[Tsukihime 1,487]

rather than tinkering around with your actual events inside the game, which is not only more time-consuming, but more prone to errors.

 

Sounds good. I've never really liked the idea of hardcoding event dialog.

I think all data should be referenced from a single location rather than being scattered across the project.

 

One concern is actually retrieving the text.

 

@actor.description = $data_text.getLine(149581)
@actor.name = $data_text.getLine(149720)

 

If it's based on string number (or offset?), does that mean if I decide to remove a line from the middle of the file, I have to go and update all references?

Edited May 23, 2012 by Tsukihime

[Kaelan 25]

The format is mainly a giant binary array of strings (with some extra data like a header, string sizes and an optional name of an audio file for dialog), so the ID is just an offset from the start of the internal string table. If you ignore the other data and use it as a giant array, it's basically just the array index.

 

 

 

Looks like this internally:

 

 

 

 

My example was hardcoding the ID just to show the simplest possible use of this. If you hardcoded the IDs for everything and did that, then yes, you'd have to update the references.

 

In my actual game, I'm using this to store the names and descriptions for all of my class, feats and spells, and each class/feat/spell entry has a number indicating what their name and description IDs are in the talk file, and those IDs are editable from the database note tags.

 

Then my actual code only has to do something like

 

  description = $data_text.getLine(@actor.classes[0].description_id)

  #....
  #display it in a menu or something

 

If you use the IDs dynamically in that way, you won't have that type of problem.

Edited May 23, 2012 by Kaelan

[Ravenith 5]

If you're making a d20 script pack, I'm TOTALLY SUPPORTING THIS!

[Chaos17 31]

This look but also complicated to use

[Ravenith 5]

Granted, it's complicated for the average RM user, but it's a must for a text heavy game. Infinity engine games would never have made it without something like this!

[Tsukihime 1,487]

Don't know how popular TLK files are but if it provides the average game developer with better text management skills then it's something they can consider adding to their resume LOL

[Kaelan 25]

It might actually be relevant if you're applying for a job in Bioware, I think they still use an updated version of it for the Dragon Age games

[Tsukihime 1,487]

Aside from translation, the tlk files can be used to separate different players as well.

For example, you have 4 actors in your game, each of whom can be controlled separately.

 

If actor1 talks to an NPC, then the NPC will say one thing.

If actor2 talks to the same NPC, then the NPC will say something else.

 

This is not something trivial like "Hey <name> how are you?" it is a completely different dialog.

 

However one obvious restriction is that your events can only be set up the same way. You can't have actor1 just saying whatever, and actor2 coming up with a choice box without explicitly setting those as separate branches.

 

But simply mapping dialog according to who is the "leader" can be done somewhat cleanly.

[Kayzee 3,815]

Hmmm, honestly it seems to me like it would be better to use a text-based format instead of a binary one, but I guess since the format already has tools for it, it isn't so bad. :3

[skitzen 0]

This is really cool! But I was wondering. I have a game I am making right now. And I am getting translators for French and German. Which both work for the game fine so far.

My game is for XP not vx ace though. :/ And I am trying to get Russian to work as a language for it.

Is there any way or anyone who has an XP script that would work like this? (if it helps each language is a separate game version so nothing gets messed up)

[Tsukihime 1,487]

The idea of using a TLK file is so that you have one version of the game, with separate TLK files for each language. This way all that matters if translating the TLK files.

[skitzen 0]

The idea of using a TLK file is so that you have one version of the game, with separate TLK files for each language. This way all that matters if translating the TLK files.

Yes I know. But the program I am using is also XP and I did not know of this until now so I would have to re enter all the text for my game. I want each language to be a separate version.

And rpg maker does not work with Russian, it comes up as: ?????????

[Tsukihime 1,487]

Can this be used in show-text and scrolling text and choice list event commands? If so, how?

[Kaelan 25]

Hmm I hadn't thought of that. I guess you wouldn't be able to, since you can't access any of the scripting from a text box. It would probably require adding something extra, like a new control character that would pull the data out for you.

 

The only problem is you wouldn't actually be able to see the Talk Table text inside the text preview window. I'm not sure what that window is calling to get its text boxes, but it doesn't seem to go through the Window_Base functions (you can try this yourself with Yanfly's message system too, none of the escape characters get converted in the preview, only in-game).

Edited November 20, 2012 by Kaelan

[Tsukihime 1,487]

That's right. However, I'm sure people will figure out the limits after some experimentation...