Class: Sass::Script::Lexer

Inherits:
Object
  • Object
show all
Includes:
Sass::SCSS::RX
Defined in:
/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb

Overview

The lexical analyzer for SassScript. It takes a raw string and converts it to individual tokens that are easier to parse.

Direct Known Subclasses

CssLexer

Defined Under Namespace

Classes: Token

Constant Summary

Instance Method Summary (collapse)

Methods included from Sass::SCSS::RX

escape_ident

Constructor Details

- (Lexer) initialize(str, line, offset, options)

Returns a new instance of Lexer

Parameters:

  • str (String, StringScanner) —

    The source text to lex

  • line (Fixnum) —

    The 1-based line on which the SassScript appears. Used for error reporting and sourcemap building

  • offset (Fixnum) —

    The 1-based character (not byte) offset in the line in the source. Used for error reporting and sourcemap building

  • options ({Symbol => Object}) —

    An options hash; see the Sass options documentation



151
152
153
154
155
156
157
158
159
160
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 151

def initialize(str, line, offset, options)
  @scanner = str.is_a?(StringScanner) ? str : Sass::Util::MultibyteStringScanner.new(str)
  @line = line
  @offset = offset
  @options = options
  @interpolation_stack = []
  @prev = nil
  @tok = nil
  @next_tok = nil
end

Instance Method Details

- (Boolean) after_interpolation?

Returns Whether or not the last token lexed was :end_interpolation.

Returns:

  • (Boolean) —

    Whether or not the last token lexed was :end_interpolation.



215
216
217
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 215

def after_interpolation?
  @prev && @prev.type == :end_interpolation
end

- (String) char(pos = @scanner.pos)

Returns the given character.

Returns:

  • (String) 


187
188
189
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 187

def char(pos = @scanner.pos)
  @scanner.string[pos, 1]
end

- (Boolean) done?

Returns Whether or not there’s more source text to lex.

Returns:

  • (Boolean) —

    Whether or not there’s more source text to lex.



208
209
210
211
212
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 208

def done?
  return if @next_tok
  whitespace unless after_interpolation? && !@interpolation_stack.empty?
  @scanner.eos? && @tok.nil?
end

- expected!(name)

Raise an error to the effect that name was expected in the input stream and wasn’t found.

This calls #unpeek! to rewind the scanner to immediately after the last returned token.

Parameters:

  • name (String) —

    The name of the entity that was expected but not found

Raises:



227
228
229
230
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 227

def expected!(name)
  unpeek!
  Sass::SCSS::Parser.expected(@scanner, name, @line)
end

- (Fixnum) line

The line number of the lexer’s current position.

Returns:

  • (Fixnum) 


29
30
31
32
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 29

def line
  return @line unless @tok
  @tok.source_range.start_pos.line
end

- (Token) next

Moves the lexer forward one token.

Returns:

  • (Token) —

    The token that was moved past



165
166
167
168
169
170
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 165

def next
  @tok ||= read_token
  @tok, tok = nil, @tok
  @prev = tok
  tok
end

- (Fixnum) offset

The number of bytes into the current line of the lexer’s current position (1-based).

Returns:

  • (Fixnum) 


38
39
40
41
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 38

def offset
  return @offset unless @tok
  @tok.source_range.start_pos.offset
end

- (Token) peek

Returns the next token without moving the lexer forward.

Returns:

  • (Token) —

    The next token



194
195
196
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 194

def peek
  @tok ||= read_token
end

- (String) str { ... }

Records all non-comment text the lexer consumes within the block and returns it as a string.

Yields:

  • A block in which text is recorded

Returns:

  • (String) 


237
238
239
240
241
242
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 237

def str
  old_pos = @tok ? @tok.pos : @scanner.pos
  yield
  new_pos = @tok ? @tok.pos : @scanner.pos
  @scanner.string[old_pos...new_pos]
end

- unpeek!

Rewinds the underlying StringScanner to before the token returned by #peek.



200
201
202
203
204
205
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 200

def unpeek!
  return unless @tok
  @scanner.pos = @tok.pos
  @line = @tok.source_range.start_pos.line
  @offset = @tok.source_range.start_pos.offset
end

- (Boolean) whitespace?(tok = @tok)

Returns whether or not there’s whitespace before the next token.

Returns:

  • (Boolean) 


175
176
177
178
179
180
181
182
 
# File '/Users/ceppstei/Projects/sass-lang/.sass/lib/sass/script/lexer.rb', line 175

def whitespace?(tok = @tok)
  if tok
    @scanner.string[0...tok.pos] =~ /\s\Z/
  else
    @scanner.string[@scanner.pos, 1] =~ /^\s/ ||
      @scanner.string[@scanner.pos - 1, 1] =~ /\s\Z/
  end
end