libstring-tokenizer-perl - simple string tokenizer

Property Value
Distribution Debian 10 (Buster)
Repository Debian Main i386
Package filename libstring-tokenizer-perl_0.06-1_all.deb
Package name libstring-tokenizer-perl
Package version 0.06
Package release 1
Package architecture all
Package type deb
Category devel::lang:perl devel::library implemented-in::perl perl
License -
Maintainer Debian Perl Group <>
Download size 13.80 KB
Installed size 37.00 KB
String::Tokenizer is a simple string tokenizer which takes a string and splits
it on whitespace. It also optionally takes a string of characters to use as
delimiters, and returns them with the token set as well. This allows for
splitting the string in many different ways.
This is a very basic tokenizer, so more complex needs should be either
addressed with a custom written tokenizer or post-processing of the output
generated by this module. Basically, this will not fill everyones needs, but
it spans a gap between simple split / /, $string and the other options that
involve much larger and complex modules.
Also note that this is not a lexical analyser. Many people confuse
tokenization with lexical analysis. A tokenizer mearly splits its input into
specific chunks, a lexical analyzer classifies those chunks. Sometimes these
two steps are combined, but not here.


Package Version Architecture Repository
libstring-tokenizer-perl_0.06-1_all.deb 0.06 all Debian Main
libstring-tokenizer-perl - - -


Name Value
perl -


Type URL
Binary Package libstring-tokenizer-perl_0.06-1_all.deb
Source Package libstring-tokenizer-perl

Install Howto

  1. Update the package index:
    # sudo apt-get update
  2. Install libstring-tokenizer-perl deb package:
    # sudo apt-get install libstring-tokenizer-perl




2016-02-29 - Lucas Kanashiro <>
libstring-tokenizer-perl (0.06-1) unstable; urgency=low
* Team upload.
[ Salvatore Bonaccorso ]
* debian/copyright: Replace DEP5 Format-Specification URL from to URL.
* Change Vcs-Git to canonical URI (git://
* Change based URIs to based URIs
[ Axel Beckert ]
* debian/copyright: migrate pre-1.0 format to 1.0 using "cme fix dpkg-
[ gregor herrmann ]
* Strip trailing slash from metacpan URLs.
[ Salvatore Bonaccorso ]
* Update Vcs-Browser URL to cgit web frontend
* debian/control: Use HTTPS transport protocol for Vcs-Git URI
[ Lucas Kanashiro ]
* Import upstream version 0.06
* Add d/u/metadata
* Drop manpage_spelling.patch, already applied by upstream
* Update years of upstream copyright
* d/copyright: drop comma from Upstream-Contact
* Update Debian packaging copyright
* Bump debhelper compatibility level to 9
* Declare compliance with Debian policy 3.9.7
* Wrap-and-sort dependencies
* Mark package as autopkgtest-able
[ gregor herrmann ]
* Drop build dependencies which are not used anymore.
2011-08-02 - Ben Webb <>
libstring-tokenizer-perl (0.05-1) unstable; urgency=low
* Initial Release. (closes: #636288)

See Also

Package Description
libstring-trim-more-perl_0.03-1_all.deb Perl module with various string trimming utilities
libstring-trim-perl_0.005-1_all.deb trim whitespace from your strings
libstring-truncate-perl_1.100602-1_all.deb Perl module to truncate long strings
libstring-tt-perl_0.3-2_all.deb Perl module to use TT to interpolate lexical variables
libstringtemplate-java_3.2.1-2_all.deb StringTemplate templating engine for Java
libstringtemplate4-java_4.0.8-2_all.deb StringTemplate templating engine for Java
libstroke0-dev_0.5.1-9_i386.deb mouse strokes library -- development files
libstroke0_0.5.1-9_i386.deb mouse strokes library -- runtime files
libstrongswan-extra-plugins_5.7.2-1_i386.deb strongSwan utility and crypto library (extra plugins)
libstrongswan-standard-plugins_5.7.2-1_i386.deb strongSwan utility and crypto library (standard plugins)
libstrongswan_5.7.2-1_i386.deb strongSwan utility and crypto library
libstrophe-dev_0.9.2-2_i386.deb Library for writing XMPP clients - development files
libstrophe0_0.9.2-2_i386.deb Library for writing XMPP clients - shared library
libstruct-compare-perl_1.0.1-4_all.deb Perl module providing recursive diff for perl structures
libstruct-dumb-perl_0.09-1_all.deb module to create simple lightweight record-like structures