Home
last modified time | relevance | path

Searched refs:rawCharacters (Results 1 – 1 of 1) sorted by relevance

/plugin/aichat/vendor/mehrab-wj/tiktoken-php/src/
DEncoder.php13 private array $rawCharacters = []; variable in TikToken\\Encoder
26 $rawCharacters = file_get_contents(__DIR__.'/../data/characters.json');
27 if (false === $rawCharacters) {
30 $this->rawCharacters = json_decode($rawCharacters, true, 512, JSON_THROW_ON_ERROR);
86 if (!isset($this->rawCharacters[$this->characterToUnicode($char)])) {
89 $resultWord .= $this->rawCharacters[$this->characterToUnicode($char)];