|
* File StreamTok02.java Copyright 1998, R.G.Baldwin
Revised 9/24/98
The program begins by creating a file containing the
sequence of ASCII character values from 32 to 126
inclusive. In addition, several extra characters are
inserted in the sequence to illustrate special behavior of
the StreamTokenizer class.
For example, a space character is inserted every seventh
character. A space character is a default word delimiter
of the StreamTokenizer class.
A double quote character is inserted several character
positions beyond the normal position of the double quote
character in the ASCII sequence to cause the characters in
between to be treated as a quoted string. The double quote
is a default delimiter for a quoted string for the
StreamTokenizer.
Likewise, a single quote character is inserted several
character positions beyond the normal position of the
single quote character in the ASCII sequence to cause the
characters in between to be treated as a quoted string.
The single quote is a default delimiter for a quoted
string for the StreamTokenizer.
Each input character is treated by the StreamTokenizer
to be one of the following:
1. A whitespace character that delimits a word.
2. An ordinary character that is not a delimiter and is
not part of either of the following.
3. A character that is part of a word.
4. A character that is part of a number.
There are also some special capabilities having to do with
the "/" character and the "*" as used in Java comments.
A method of the class was used to cause the "/" character
to be treated as an ordinary character.
A method of the class was used to cause the s, j, k, and l
characters to be treated as whitespace characters to
delimit words.
A method of the class was used to cause the ";" and "="
character to be treated a word characters.
The value returned by the nextToken() method indicates
how a character is being interpreted according to the
possibilities listed above. A switch statement was used
to analyze the return value and take the appropriate
action based on that return value.
A public instance variable of the class contains the same
information and can be used for similar purposes.
The file was read and parsed by an object of the
StreamTokenizer class. By using a combination of the
return value from the nextToken() method and the
instance variable, a printout wa
1000
s produced showing how the
file was parsed by the object.
Finally, the file was read again and displayed as numeric
and character data to confirm its contents and make it
possible to compare those contents with the behavior of the
parser.
Program output:
Start the program and write a file
returnValue=33 ttype=33 char = !
Quoted string= #$% ttype=34 char = "
returnValue=38 ttype=38 char = &
Quoted string=() ttype=39 char = '
returnValue=42 ttype=42 char = *
returnValue=43 ttype=43 char = +
returnValue=44 ttype=44 char = ,
TT_NUMBER -0.0 ttype=-2
returnValue=47 ttype=47 char = /
TT_NUMBER 0.0 ttype=-2
TT_NUMBER 1234567.0 ttype=-2
TT_NUMBER 89.0 ttype=-2
returnValue=58 ttype=58 char = :
TT_WORD ;<= ttype=-3
returnValue=62 ttype=62 char = >
returnValue=63 ttype=63 char = ?
returnValue=64 ttype=64 char = @
TT_WORD ABCDE ttype=-3
TT_WORD FGHIJKL ttype=-3
TT_WORD MNOPQRS ttype=-3
TT_WORD TUVWXYZ ttype=-3
returnValue=91 ttype=91 char = [
returnValue=92 ttype=92 char = \
returnValue=93 ttype=93 char = ]
returnValue=94 ttype=94 char = ^
returnValue=95 ttype=95 char = _
returnValue=96 ttype=96 char = `
TT_WORD a ttype=-3
TT_WORD bcdefgh ttype=-3
TT_WORD i ttype=-3
TT_WORD mno ttype=-3
TT_WORD pqr ttype=-3
TT_WORD tuv ttype=-3
TT_WORD wxyz ttype=-3
returnValue=123 ttype=123 char = {
returnValue=124 ttype=124 char = |
returnValue=125 ttype=125 char = }
returnValue=126 ttype=126 char = ~
Display file data
32 33 ! 34 " 32 35 #
36 $ 37 % 34 " 38 & 39 '
40 ( 41 ) 32 39 ' 42 *
43 + 44 , 45 - 46 . 47 /
48 0 32 49 1 50 2 51 3
52 4 53 5 54 6 55 7 32
56 8 57 9 58 : 59 ; 60 <
61 = 62 > 32 63 ? 64 @
65 A 66 B 67 C 68 D 69 E
32 70 F 71 G 72 H 73 I
74 J 75 K 76 L 32 77 M
78 N 79 O 80 P 81 Q 82 R
83 S 32 84 T 85 U 86 V
87 W 88 X 89 Y 90 Z 32
91 [ 92 \ 93 ] 94 ^ 95 _
96 ` 97 a 32 98 b 99 c
100 d 101 e 102 f 103 g 104 h
32 105 i 106 j 107 k 108 l
109 m 110 n 111 o 32 112 p
113 q 114 r 115 s 116 t 117 u
118 v 32 119 w 120 x 121 y
122 z 123 { 124 | 125 } 32
126 ~ End of program
Tested using JDK 1.1.6 under Win95.
**********************************************************/
import java.io.*;
class StreamTok02{
public static void main(String[] args)
{
System.out.println(
&n
1000
bsp; "Start the program and write a file");
try{
//Instantiate and initialize an output file stream
// object.
FileOutputStream outFile =
new FileOutputStream("junk.txt");
//Write a series of bytes to the file and close it
for(int cnt = 32; cnt < 127; cnt++){
if(cnt%7 == 0){
outFile.write(' ');//every seventh char is space
}//end if
//Force double quotes around several char following
// the double quote in the ASCII sequence
if(cnt == '&') outFile.write('\"');
//Force single quotes around several char following
// the single quote in the ASCII sequence
if(cnt == '*') outFile.write('\'');
outFile.write((char)cnt);//write the byte
}//end for loop
outFile.close();
//Instantiate and initialize an input stream
FileInputStream inFile =
new FileInputStream("junk.txt");
Reader rdr = new BufferedReader(
new InputStreamReader(inFile));
//Instantiate a StreamTokenizer object
StreamTokenizer strTok = new StreamTokenizer(rdr);
//Convert the "/" character to an ordinary char
strTok.ordinaryChar('/');//forward slash
//Make the lower-case s, j, k, and l, whitespace char.
// They will function as word delimiters in the parsed
// data stream.
strTok.whitespaceChars('s','s');
strTok.whitespaceChars('j','l');
//Make the ;<= characters to be word characters
strTok.wordChars(';','=');
//Loop getting sequential tokens from the file,
// interpreting, and displaying those tokens
int returnValue = strTok.nextToken();//priming read
while(returnValue != StreamTokenizer.TT_EOF){
//Terminate the loop on end of file
switch(returnValue){//determine the type of token
case StreamTokenizer.TT_EOF ://shouldn't be here
System.out.print("TT_EOF ");break;
case StreamTokenizer.TT_EOL ://end of line
System.out.print("TT_EOL ");break;
case StreamTokenizer.TT_NUMBER ://a number
System.out.print("TT_NUMBER " + strTok.nval);
break;
case StreamToke
1000
nizer.TT_WORD ://a word
System.out.print("TT_WORD " + strTok.sval);break;
case '\"' ://a double quote
System.out.print("Quoted string=" + strTok.sval);
break;
case '\'' ://a single quote
System.out.print("Quoted string=" + strTok.sval);
break;
default:System.out.print(//none of the above
"returnValue=" + returnValue);
}//end switch
//Use ttype variable to display additional info.
System.out.print(" ttype=" + strTok.ttype + " ");
if(strTok.ttype >= 0)//neg is eof, eol, number, word
//display the character
System.out.println("char = " + (char)strTok.ttype);
else System.out.println();//new line on eol, etc.
returnValue = strTok.nextToken();//get next token
}//end while loop
inFile.close();//close the file
System.out.println(); //new line
System.out.println("Display file data");
//Open the file again for simple read and display
inFile = new FileInputStream("junk.txt");
int data;
int cnt = 0;
while( (data = inFile.read()) != -1){
System.out.print("" + data + " ");
System.out.print((char)data + " ");
if(cnt++ == 4){//new line every fifth read
System.out.println();//new line
cnt = 0;//reinitialize the counter
}//end if
}//end while
inFile.close();//close file again
}catch(IOException e){System.out.println(e);}
System.out.println("End of program");
}// end main
}//end class StreamTok02 definition
|
גולשים מקוונים: 5
