The Language Pack Concept

From eXo Wiki
Revision as of 13:18, 5 April 2022 by Timber (talk | contribs) (→‎eXo\util\!french)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Technical Specs

Core concepts overview

Language Packs are “add-ons”. They need eXoDOS full installed. Yet they are designed to be as separate as possible to avoid conflicts, overwrites and complicated merges that require parsing and validating existing data, at the cost of some potential data duplication.

  • Each language is a separate pack. Users can install several as they do not overlap
  • Each language pack has its own release schedule, asynchronous from eXoDOS or other language packs
  • Each language pack includes its own XML that its installer will merge in eXoDOS’ “MS-DOS.xml” leveraging the existing “merge.bat” util
  • Each language pack includes its own Playlist (xml) with all games included in the language pack and all games, which are already included in eXoDOS and contain beside English among other also this language
  • eXoDOS start scripts (batch files) always check if there are different language versions available and let the user select their prefered language
  • For games that do not exist in base-eXoDOS, a new game entry is appended to MS-DOS.xml
  • Multilanguage games (that may or may not exist in eXoDOS already) where language is set during installation or setup or through a change of files get their own standalone preconfigured entry, which implies duplication of sources
  • Multilanguage games (that exist in eXoDOS already) where language is set during startup or ingame are linked to the base-eXoDOS entry in the language playlist

Structure

As separate add-ins, language pack files are stored in their own folder structure that plugs to eXoDOS (examples are for the French language pack)

xml\french

  • Contains the LB platform XML files (“french.xml” and “frenchfamily.xml”) that will be

merged into MS-DOS.xml in the same folder by the language pack setup.

eXo\eXoDOS\!french

  • Contains all localized game zips and is the destination for installed games. Functional equivalent of “ \eXo\eXoDOS” for regular eXoDOS

eXo\eXoDOS\!dos\!french

  • Contains individual game folders that include localized
    • install.bat
    • game name (YEAR).bat
    • dosbox.conf
    • exception.bat (optional)
  • Each game folders contain an “Extras” subfolder for localized manual and other documents

as well as a localized Alternate Launcher.bat

  • Functional equivalent of “X:\eXoDOS\eXo\eXoDOS\!dos” for regular eXoDOS

Images\MS-DOS\<image type>\France

  • Localized game images should be stored in a regional subfolder for easier packing.

LaunchBox already has pre-configured folders such as France and Germany. A new folder should be created if necessary.

After discussion with eXo, this requires a little more work as it is probable LB just loads images based on its “region” settings. Open question is: are multiple regions supported for a single game, and if yes, will it fetch images in all region folders? This is a problem for existing games mainly, as new games (=not existing in eXoDOS) will be added with the proper region. I tested the whole image thing in LB. The regions currently set in Launchbox are „North America“ and „United States“ in this order. The Language pack setup has to add the language pack region and set the order to „<language pack language>“, „United States“, „North America“. LB loads the Images in this order then from the folders

eXo\dosbox

  • Contains a localized config.bat file (renamed to config_FR.bat for French), for global DOSBox

settings management

Is this really needed? It would be called in the eXo/util/!german/Launch.bat

eXo\util\!french

  • Contains all localized util files:
    • dosbox.txt (with the matching table for the games<->dosbox version
    • texts.txt (language text lines)
    • multiplayer.txt (with the matching table "game has multiplayer options")
    • openFolder.bat (the batch file to open the extras folder for the language pack (see "Launchbox integration")
    • Launch.bat (the batch that launches the games)
    • IP.bat (the batch that sets the IP txt files if the game supports multiplayer)

eXo\LanguagePacks\french (YET TO BE DEFINED – Placeholder)

  • Contains setup batch + other necessary files for language packs setup and management.

My suggestion would be to include localized setup batches in every language pack in their own folder, then instruct users to extract the language pack over their existing eXoDOS installation, then browse to “LanguagePacks\French” and run setup_FR.bat there, for example. This way in the case of several language packs installed, the setup files do not overwrite each other in a different language, and we do not have to worry about asking the user what languages he wants the setup in. The language of the setup is always in the language of the pack it belongs to. Yet if it finds several language XML’s, it installs all of them

Setup and update

The full setup mechanism is still to be precisely defined at this stage.

Existing base concepts:

  • Extraction of the xml files to the xml folder
  • Extraction of all localized game zips to the localized game folder
  • Extraction of all game files and extras to the localized “!dos” folder
  • Extraction of all pictures to the region-specific images folders
  • Extraction of the localized config batch file
  • Extraction of the localized util-folder files

XML Merging:

  • Merge.bat parses all xml files in the xml folder and identifies installed language pack(s)
  • Asks the user if he wants adult games or not to select the proper source XML from the

language pack(s) and the proper destination XML from the base eXoDOS install

  • Merges the appropriate language pack(s) xml files into the chosen base xml.
    • Adult games included: merges french.xml (+other installed language packs) into base

MS-DOS.xml, copy to Data\Platforms

    • Adult games excluded: merges frenchfamily.xml (+other installed language packs)

into dosfamily.xml and copy to Data\Platforms as MS-DOS.xml

This implies every time a language pack is installed (be it for the first time or for an update), it creates a new clean MS-DOS.xml, overwriting potential manual changes. that’s the same with base-eXoDOS, it would overwrite the xml – I think manual changes shouldn‘t be considered

Bat files localization

Game install and launch bat files need to have their “echo” instructions localized for a better user experience. A template of files is created for each language. The download on demand mechanism is stripped out, as language packs will only be released as full packages (no “lite” releases). To support foreign languages accents and special characters, the codepage of batch files needs to be changed to Unicode (UTF-8). This is done by adding these lines to the beginning of batch files to call a codepage change to 65001 (ref: https://docs.microsoft.com/en-us/windows/win32/intl/code-page-identifiers)

I think we should stick with ae/oe/ue and no special characters … We need to use special editor-config to edit the batch files. A „normal“ edit with notepad would „destroy“ the file – I think that’s a high risk for failures? - needs to be discussed but maybe every language pack admin can decide himself?


@echo off
REM call Codepage 65001 for UTF-8 accents support
@chcp 65001
cls

The files that need translation are:

  • Install.bat (in every “!dos\!french\”-game folder - the file is always the same and can just be copied to every folder)
  • Gamename (year).bat (in every “!dos\!french\”-game folder - the file is always the same and can just be copied to every folder)
  • exception.bat (optional in the game folder ("eXo\eXoDOS\!dos\!french\..."), mostly for scummvm games
  • Alternate Launcher.bat (in every game folder “Extras” subfolder)
  • Config.bat (only once in “eXo\Dosbox”, renamed to config_fr.bat)
  • Launch.bat (only once in "eXo\util", moved to "eXo\util\!french")

Games packaging and localized names

Games are packaged in the same format as other eXoDOS games: GameName (year).zip When an official localized name exists for a game, this should be used instead of the English name.

currently this works because the eXoDOS batch "searches" for the game folder, so this has to be named the same as in eXoDOS But: the current way the games that only have extras but no language specific game in the language pack won't work. Because they also have the game folder (where the extras are in), the eXoDOS batch will find it and let the user choose the language, then tries to start the game name (year).bat, which isn't there... So I think it needs to be the same file name as in eXoDOS (as long as the game is already in eXoDOS) and the eXoDOS batch has to be changed to look for the game name (year).bat instead of only the game name folder

LaunchBox integration

Three additional applications need to be merged into existing games, with the language as prefix to sort them out properly:

  • Standard game
  • Alternate Launcher (pixel perfect & shader options)
  • Open Extras folder (for localized manuals and docs)

Example for Space Quest IV French:

“Open Folder.bat” is a simple batch: “explorer.exe .” that opens explorer from the folder the batches is run from. These new entries should also be localized themselves for consistency. How it looks in the contextual menu:

What about new games that do not exist yet in eXoDOS? We just merge a full game entry in the french.xml instead of an additional application? Unlikely scenario – what happens if two language packs both insert the same game that doesn’t exist in eXoDOS (ie, a game that would only exist in French and German but not English). Borderline, I know... It’s part of the „Add a game to the language pack“ section

Language Playlists

Every language pack comes with it’s own language games playlist. For example: French_Playlis.xml It contains all the games of the language pack, whether they were added by the language pack or they were already in base-eXoDOS On how the playlist is created look at the „lists and automatism“ section

Lists and automatism

We created scripts to export from our spreadsheet lists to XML files for Launchbox (and to be merged with the eXoDOS base pack XMLs)

Google spreadsheet

The google spreadsheet (https://docs.google.com/spreadsheets/d/1x19Yw1ZnRYF1vwTXd9PTGxqZBQkkEtPmjiqlUUdq1oY/edit?usp=sharing) contains a sheet for every language. The eXoID column is filled automatically by script from the master sheet.

Every language pack has it's own spreadsheet. Here's the one for the German language pack (GLP): https://docs.google.com/spreadsheets/d/1C1nRFVZkBHWodF7tOHm5hWA0YV45svLEQCUMCUl70dA It needs a sheet called "*LP" (* for the language-letter, example: "GLP" for the German pack, "FLP" for the French pack, "PLP" for the polish pack ...), a sheet called "extras" and, if magazines shall be included, a sheet called "magazines" All other sheets are optional.

xLP sheet

This sheet contains the games list. From this list the final XML file to be merged with eXoDOS' "MS-DOS.XML" is automatically created. Therefor it needs the following columns:

  • A ("eXoID") = cross matching to the eXoDOS master table (if the game is already in eXoDOS)
  • D ("Game") = it’s the title of the game in eXoDOS
  • E ("LP Name") = it’s the title of the game in the language pack (= will be set as an alternate title in LaunchBox)
  • G ("batchname") = it's the name of the game batch (game name (year)), important for the creation of the /eXo/util/!<language>/dosbox.txt
  • H ("Folder") = it’s the folder of the game in eXoDOS. It must be the same in the language pack for the batch files to work correct. eXo has build a language switch that checks the language-folder (eXo/eXoDOS/!dos/!german) for a game folder and lets the user select the language version to start the game with checkup should be changed to the game name (year) batch file ... see "Games packaging and localized names"
  • T ("variant") = that column decides, which variant is used for the game (see chapter 4)
  • W ("dosbox") = contains the call for the dosbox-version - the eXo/util/!german/dosbox.txt is filled from that column (and the batchname column)
  • AC ("LB-ID") = it’s the ID LaunchBox uses for connecting the game with it’s additional applications, alternate names and playlists in the xml files. The automation-script needs it to create the xml files for the language pack - for games that are already in eXoDOS it has to be the same as in eXoDOS base package (to link the language pack's additionalApplication to the original entry). For language pack exclusive games that aren't in eXoDOS base package, the LB-ID can be created with every UUID v4 generator (like https://www.uuidgenerator.net/version4)

extras sheet

This sheet contains the list of extras for a game. The extra files themselves are placed in the folder: /eXo/eXoDOS/!dos/!<language>/<game folder>/Extras The rows in this sheet are needed for the automation to create the "Additional Application" entries in the XML files

  • A-C (eXoID, LB-ID, game name) = these columns are filled automatically by writing a foldername in column D
  • D (folder) = here the game folder name has to be used. from this the macros will fill columns A to C automatically. It's needed to cross match the entries with the game entries for creating the XML files afterwards
  • E (filename) = the name + extension of the extras file (no folders needed)
  • F (title) = the title or name of the File - it will be used for the Extras entry in the right-click menu of Launchbox
  • G (english) = the title or name in english (not used yet, but maybe we decide to switch the name in Launchbox to this, starting with "(<LANGUAGE>) "?)

not needed atm because we only add an "Additional Application" to open the Extras folder of the game. but should be filled out anyway because maybe this changes someday because of possibility of creating cascading menus in LB for example

magazines sheet

Here all game reviews, previews, hints ... that were found in magazines can be placed to enable direct linking of them to a game in the language pack. The automation script will create an AdditionalApplication entry using sumatra pdf to open the magazin on the correct page for the game review (or preview or hints or whatever)

the Magazines are placed in the folder /eXo/Magazines/!german/<Magazine-name>

guides sheet

same as magazines sheet but for game guides

the guides are placed in the folder /eXo/Guides/!german/

Language pack xml creation

The automation script creates the language pack xmls (french.xml and frenchfamily.xml). They contain all the entries that will be added to the MS-DOS.xml when the language pack is installed It’s based on the google spreadsheet (see "Google spreadsheet")

insert script listings here


Adding a game to the language pack

Procedure

image missing  

Variants

The variants are set in the google spreadsheet (see "Google spreadsheet") on sheet "xLP" in column "T (variant)"

new

the game is not in eXoDOS, only in the language pack

  • <gamefolder> has to be named (unique)
  • Game Name (XXXX) has to be set (in most cases mobygames can be uses as source)
  • Manual is placed in /Manuals/MS-DOS/<Launchbox-gamename>.pdf
  • Music file is placed in /Music/MS-DOS/<Launchbox-gamename>.mp3
  • Video is placed in /Videos/MS-DOS/<Launchbox-gamename>.mp4
  • Game files have to be placed in /eXo/eXoDOS/!german/<gamefolder> (create folder if not exists)
  • create Game Name (XXXX).ba1 in /eXo/eXoDOS/!german/<gamefolder> (create folder if not exists)
  • zip the /eXo/eXoDOS/!german/<gamefolder> and rename the zip to Game Name (XXXX).zip
  • install.bat, Game Name (XXXX).bat and dosbox.conf have to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder> (create folder if not exists)
  • exception.bat (if needed) has to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder> (create folder if not exists)
  • Extras have to be placed in /eXo/eXoDOS/!dos/!german/<game folder>/Extras (create folder if not exists)
  • Images (at least the types "box – front", "box – back", "disc", "screenshot gametitle" and "screenshot gameplay") have to be placed in /Images/MS-DOS/<type>/German/<Launchbox-gamename>-xx.<ext> (xx = counter starting with 01, <ext> = file extension of the image file)
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set „variant“ column to „new“
    • add extras rows in the extras sheet (they will be automatically added to the XML for the merge

overview:

  • game gets its own new entry in Launchbox (LBID will be set during XML creation)
  • game is added to the German playlist
  • extras will be linked as additional applications to that LB entry like in eXoDOS
  • Manual is automatically set by LB
  • Music is automatically set by LB
  • Video is automatically set by LB
  • Images are automatically set by LB

extras

the game is already in eXoDOS and only extras (manuals, documents …) will be added by the language pack

  • Extras and Manual have to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder>/Extras (create folder if not exists)
  • Images (at least the types "box – front", "box – back", "disc", "screenshot gametitle" and "screenshot gameplay") have to be placed in /Images/MS-DOS/<type>/German/<Launchbox-gamename>-xx.<ext> (xx = counter starting with 01, <ext> = file extension of the image file)
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set „variant“ column to „extras“
    • add extras rows in the extras sheet (they will be automatically added to the XML for the merge

overview:

  • Game is added to the German playlist
  • for German extras and manual there will be an additional application to the eXoDOS entry „Deutsche Extras und Dokumente“ opening the German extras folder
  • German images are automatically set by LB


add

the game is already in eXoDOS but it’s not in German - so German game version will be added by the language pack

  • Game files have to be placed in /eXo/eXoDOS/!german/<gamefolder> (create folder if not exists)
  • create Game Name (XXXX).ba1 in /eXo/eXoDOS/!german/<gamefolder> (create folder if not exists)
  • zip the /eXo/eXoDOS/!german/<gamefolder> and rename the zip to Game Name (XXXX).zip
  • Install.bat, Game Name (XXXX).bat and dosbox.conf have to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder> (create folder if not exists)
  • exception.bat (if needed) has to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder> (create folder if not exists)
  • Extras and Manual have to be placed in eXo/eXoDOS/!dos/!german/<gamefolder>/Extras (create folder if not exists)
  • Images (at least the types "box – front", "box – back", "disc", "screenshot gametitle" and "screenshot gameplay") have to be placed in /Images/MS-DOS/<type>/German/<Launchbox-gamename>-xx.<ext> (xx = counter starting with 01, <ext> = file extension of the image file)
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set „varaiant“ column to „add“
    • add extras rows in the extras sheet (they will be automatically added to the XML for the merge

overview:

  • Game is added to the German playlist
  • For German extras and manual there will be an additional application to the eXoDOS entry „Deutsche Extras und Dokumente“ opening the German extras folder
  • Images are automatically set by LB

 

keep

the game is already in eXoDOS and it is German language only - so language pack only needs to add the playlist entry

  • Extras, Images, Manuals can all be added to the eXoDOS base package
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set „variant“ column to „keep“
    • IMPORTANT: Set column R = eXoDOS to „Y“ (= the game is only added to the playlist.xml for the language pack)

overview:

  • Game will only be added to the German playlist

 

mlingame

the game is already in eXoDOS and it's a multilanguage version with language choice ingame (menu or key combination)

  • create Game Name (XXXX).ba1 in /eXo/eXoDOS/!german/<gamefolder> (create folder if not exists)
  • zip the /eXo/eXoDOS/!german/<gamefolder> and rename the zip to Game Name (XXXX).zip
  • Game Name (XXXX).bat has to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder> (create folder if not exists) with German info and steps on how to change the language ingame and loop back to the eXoDOS start batch
  • Extras and Manual have to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder>/Extras (create folder if not exists)
  • Images (at least the types "box – front", "box – back", "disc", "screenshot gametitle" and "screenshot gameplay") have to be placed in /Images/MS-DOS/<type>/German/<Launchbox-gamename>-xx.<ext> (xx = counter starting with 01, <ext> = file extension of the image file)
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set "variant" column to „mlingame“
    • add extras rows in the extras sheet (they will be automatically added to the XML)

overview:

  • Game is added to the German playlist
  • For German extras and manual there will be an additional application to the eXoDOS entry „Deutsche Extras und Dokumente“ opening the German extras folder
  • Images are automatically set by LB

 

mlstartup

the game is already in eXoDOS and it's a multilanguage version with language choice at startup (either in the eXoDOS batch files or at game startup)

  • Extras and Manual have to be placed in /eXo/eXoDOS/!dos/!german/<gamefolder>/Extras (create folder if not exists)
  • Images (at least the types "box – front", "box – back", "disc", "screenshot gametitle" and "screenshot gameplay") have to be placed in /Images/MS-DOS/<type>/German/<Launchbox-gamename>-xx.<ext> (xx = counter starting with 01, <ext> = file extension of the image file)
  • In German language pack spreadsheet:
    • fill out the columns for the game (or add new row if game is not in yet) -> important: set „variant“ column to „mlstartup“
    • add extras rows in the extras sheet (they will be automatically added to the XML )

overview:

  • Game is added to the German playlist
  • For German extras and manual there will be an additional application to the eXoDOS entry „Deutsche Extras und Dokumente“ opening the German extras folder
  • Images are automatically set by LB
Retrieved from "https://wiki.retro-exo.com/index.php?title=The_Language_Pack_Concept&oldid=628"