[Skip Navigation] [CSUSB] / [CNS] / [CSE] / [R J Botting] / [Samples] / python.syntax
[Index] [Contents] [Source Text] [About] [Notation] [Copyright] [Comment/Contact] [Search ]
Sat Jun 11 15:03:01 PDT 2011

Contents


    Syntax of the Python Programming Language

      Python Notes

      [ python.html ]

      Source

      Translated into XBNF/MATHS from [ grammar.txt ] (November 2010)

      I would like to thank Martin v. Löwis for correcting the syntax for expressions and lists in November 2010.

      Macros used in this Syntax

    1. For x, O(x)::= (|x), optional x.
    2. #(x)::= O(N(x)), any number of x including none.
    3. N(x)::=x | x N(x), any number of x.

    4. List(x)::=x #("," x), list of x separated by a comma.
    5. OC::=O(","), optional comma, found at the end of a line.

      Lexemes

    6. identifier::lexeme= (letter|"_") #(letter | digit | "_").

    7. letter::lexeme= lowercase | uppercase.

    8. lowercase::lexeme= "a".."z".

    9. uppercase::lexeme= "A".."Z".

    10. digit::lexeme= "0".."9".

    11. stringliteral::lexeme= O(stringprefix)(shortstring | longstring).

    12. stringprefix::lexeme= "r" | "u" | "ur" | "R" | "U" | "UR" | "Ur" | "uR".

      "r" and "R" indicate "raw" with no escape sequences. "u" and "U" indicate Unicode.

    13. shortstring::lexeme= quote #( shortstringitem(quote)) quote | quotes #( shortstringitem(quotes) quotes.
    14. quote::lexeme= "'", used in shortstring, and threequotes.
    15. quotes::lexeme= "\"", double quotation symbol, used in shortstring.

    16. longstring::lexeme= "'''" #( longstringitem) "'''" | three_quotes #( longstringitem) three_quotes, inside a longstring there can be newlines and escape sequence but not the three quot or quotes string that starts and finishes it.
    17. three_quotes::lexeme=quotes quotes quotes, used in longstring.

    18. shortstringitem(x)::lexeme= shortstringchar(x) | escapeseq .
    19. longstringitem::lexeme= longstringchar | escapeseq.

    20. shortstringchar::lexeme= char ~ ("\\" | x | NEWLINE ). any source character except "\" or newline or the quote.

    21. longstringchar::lexeme= char~ "\\".

    22. escapeseq::lexeme= "\\" char.

    23. longinteger::lexeme= integer ("l" | "L").

    24. integer::lexeme= decimalinteger | octinteger | hexinteger.

    25. decimalinteger::lexeme= nonzerodigit #( digit) | "0".

    26. octinteger::lexeme= "0" N(octdigit).

    27. hexinteger::lexeme= "0" ("x" | "X") N(hexdigit).

    28. nonzerodigit::lexeme= "1".."9".

    29. octdigit::lexeme= "0".."7".

    30. hexdigit::lexeme= digit | "a".."f" | "A".."F".

    31. floatnumber::lexeme= pointfloat | exponentfloat.

    32. pointfloat::lexeme= O(intpart) fraction | intpart ".".

    33. exponentfloat::lexeme= (intpart | pointfloat) exponent.

    34. intpart::lexeme= digits.
    35. digits::lexeme=N(digit).

    36. fraction::lexeme= "." digits.

    37. exponent::lexeme= ("e" | "E") O("+" | "-") digits.

    38. imagnumber::lexeme= (floatnumber | intpart) ("j" | "J").

    39. atom::lexeme= identifier | literal | enclosure.

      Expressions and Forms

    40. enclosure::= parenth_form | list_display | generator_expression | dict_display | string_conversion | yield_atom.

    41. literal::= stringliteral | integer | longinteger | floatnumber | imagnumber.

    42. parenth_form::= "(" O(expression_list) ")".

    43. list_display::= "[" O(expression_list | list_comprehension) "]".
    44. start_list::lexeme= "[".
    45. end_list::lexeme= "]".
       		[ cat,dog,wombat]
       		[1, 4, 9]
       		[i*i for i in [1,2,3]]
      Square brackets are also used in subscription and slicing.

    46. list_comprehension::= expression list_for.

    47. list_for::= "for" target_list "in" old_expression_list O(list_iter).

    48. old_expression_list::= old_expression O(N("," old_expression) OC).

    49. list_iter::= list_for | list_if.

    50. list_if::= "if" old_expression O(list_iter).

    51. generator_expression::= "(" expression genexpr_for ")".

    52. genexpr_for::= "for" target_list "in" or_test O(genexpr_iter).

    53. genexpr_iter::= genexpr_for | genexpr_if.

    54. genexpr_if::= "if" old_expression O(genexpr_iter).

    55. dict_display::= "{" O(key_datum_list) "}".
    56. start_dictionary::lexeme= "{".
    57. end_dictionary::lexeme= "}".
       		{ "cat":1, "dog":2 }

    58. key_datum_list::= List(key_datum) OC.

    59. key_datum::= expression ":" expression.

    60. string_conversion::= "" expression_list "".

    61. yield_atom::= "(" yield_expression ")".

    62. yield_expression::= "yield" O(expression_list).

    63. primary::= atom | attributeref | subscription | slicing | call.

    64. attributeref::= primary "." identifier.
    65. attributor::lexeme= ".".

    66. subscription::= primary "[" expression_list "]".

    67. slicing::= simple_slicing | extended_slicing.

    68. simple_slicing::= primary "[" short_slice "]".

    69. extended_slicing::= primary "[" slice_list "]" .

    70. slice_list::= List(slice_item ) OC.

    71. slice_item::= expression | proper_slice | ellipsis.

    72. proper_slice::= short_slice | long_slice.

    73. short_slice::= O(lower_bound) ":" O(upper_bound).
    74. slicer::lexeme= ":".

    75. long_slice::= short_slice ":" O(stride).

    76. lower_bound::= expression.

    77. upper_bound::= expression.

    78. stride::= expression.

    79. ellipsis::lexeme= "..."

    80. call::= primary "(" O(argument_list OC | expression genexpr_for) ")".

    81. argument_list::= positional_arguments O("," keyword_arguments) O("," "*" expression) O("," "**" expression) | keyword_arguments O("," "*" expression) O("," "**" expression) | "*" expression O("," "**" expression) | "**" expression.

    82. positional_arguments::= List(expression).

    83. keyword_arguments::= List(keyword_item ).

    84. keyword_item::= identifier "=" expression.

      Expressions

    85. power::= primary O("**" u_expr).

    86. u_expr::= power | unary_operator u_expr.
    87. unary_operator::lexeme= "-" | "+" | "~".

    88. m_expr::= u_expr | m_expr "*" u_expr | m_expr "//" u_expr | m_expr "/" u_expr | m_expr "%" u_expr.
    89. multiplier::lexeme= "*" .
    90. int_divisor::lexeme= "//".
    91. float_divisor::lexeme= "/", warning -- in Python 2.* dividing two ints like this "i/j" is done with int division, as in C and C++.
    92. modulus::lexeme= "%".

    93. a_expr::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr.
    94. adder::lexeme= "+".
    95. subtractor::lexeme= "-".

    96. shift_expr::= a_expr | shift_expr shifter a_expr.
    97. shifter::=left_shift | right_shift.
    98. left_shift::lexeme="<<".
    99. right_shift::lexeme=">>".

    100. and_expr::= shift_expr | and_expr "&" shift_expr.
    101. and_op::lexeme="&".

    102. xor_expr::= and_expr | xor_expr "^" and_expr.
    103. xor_op::lexeme= "^".

    104. or_expr::= xor_expr | or_expr "|" xor_expr.
    105. or_op::lexeme= ":".

    106. comparison::= or_expr #( comp_operator or_expr ).

    107. comp_operator::= "<" | ">" | "==" | ">=" | "<=" | "<>" | "!=" | "is" O("not") | O("not") "in".

    108. expression::= conditional_expression | lambda_form | expression_list. A conditional expression evaluates a test and picks a subexpression to evaulate to give a result. A lambda_form creates an un-named function. An expression list evaluates to a list of values that can be accessed one after another.

    109. old_expression::= or_test | old_lambda_form.

    110. conditional_expression::= or_test O("if" or_test "else" expression).
    111. if::lexeme.

    112. or_test::= and_test | or_test "or" and_test.
    113. or::lexeme.

    114. and_test::= not_test | and_test "and" not_test.
    115. and::lexeme.

    116. not_test::= comparison | "not" not_test.
    117. not::lexeme.

    118. lambda_form::= "lambda" O(parameter_list)":" expression.
    119. lambda::lexeme.

    120. old_lambda_form::= "lambda" O(parameter_list)":" old_expression.

    121. expression_list::= List(expression ) OC.

      Statements

    122. simple_stmt::= expression_stmt | assert_stmt | assignment_stmt | augmented_assignment_stmt | pass_stmt | del_stmt | print_stmt | return_stmt | yield_stmt | raise_stmt | break_stmt | continue_stmt | import_stmt | global_stmt | exec_stmt.

    123. expression_stmt::= expression_list.

    124. assert_stmt::= "assert" List(expression ).
    125. assert::lexeme.

    126. assignment_stmt::= N(target_list "=") (expression_list | yield_expression).
    127. assigner::lexeme= "=".

    128. target_list::= List(target ) OC.

    129. target::= identifier | "(" target_list ")" | "[" target_list "]" | attributeref | subscription | slicing.

    130. augmented_assignment_stmt::= target augop (expression_list | yield_expression).

    131. augop::= "+=" | "-=" | "*=" | "/=" | "%=" | "**=" | ">>=" | "<<=" | "&=" | "^=" | "|=".

    132. pass_stmt::= "pass".
    133. pass::lexeme.

    134. del_stmt::= "del" target_list.
    135. del::lexeme.

    136. print_stmt::= "print" (O(List(expression ) OC | ">>" expression O(N("," expression) OC).
    137. print::lexeme.
       	print a,b,c
       	print a,b,c,
       	print
      The print statement outputs the values of the expressions to the users terminal. The print statement can end with a comma(",") which indicates the next output is separated by a space, otherwise the next output is output on a new line. An empty print outputs a line therefore.

    138. return_stmt::= "return" O(expression_list).
    139. return::lexeme.
       	return i+1;
      This returns the value of the expression as the value of the function. The next call of the function starts from the beginning again.

    140. yield_stmt::= yield expression.
    141. yield::lexeme.
       	yield i+1;
      This returns the value of the expression as the value of the function, but the next time the function is called it starts from the next statement.

    142. raise_stmt::= "raise" O(expression O("," expression O("," expression))). throw an exception... terminates surrounding blocks and functions until the exception is caught and handled.
    143. raise::lexeme.

    144. break_stmt::= "break". Break out of loops.
    145. break::lexeme.

    146. continue_stmt::= "continue". Go to end of loop body.
    147. continue::lexeme.

      Modules

    148. import_stmt::= "import" module optional_as_name #( "," module optional_as_name ) | "from" relative_module "import" identifier optional_as_name #( "," identifier optional_as_name ) | "from" relative_module "import" "(" identifier optional_as_name #( "," identifier optional_as_name ) OC ")" | "from" module "import" "*".
    149. import::lexeme, used in an import_stmt.

    150. optional_as_name::=O("as" name).
    151. as::lexeme.

    152. module::= #(identifier ".") identifier.

    153. relative_module::= #(".") module | N(".").

    154. name::= identifier.

    155. global_stmt::= "global" List(identifier ).
    156. global::lexeme, used in a global_stmt.

      Compound Statements

    157. exec_stmt::= "exec" or_expr O("in" expression O("," expression)).
    158. exec::lexeme, used in exec_stmt.

    159. compound_stmt::= if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef.

    160. suite::= stmt_list NEWLINE | NEWLINE INDENT N(statement) DEDENT.

    161. statement::= stmt_list NEWLINE | compound_stmt.

    162. stmt_list::= simple_stmt #(";" simple_stmt) O(";").

    163. if_stmt::= "if" expression ":" suite #( "elif" expression ":" suite ) O("else" ":" suite).
    164. if::lexeme, in if_stmt.
    165. elif::lexeme, in if_stmt.
    166. else::lexeme, in if_stmt, while_stmt, for_stmt, ...

    167. loop::=while_stmt | for_stmt

    168. while_stmt::= "while" expression ":" suite O("else" ":" suite).
    169. while::lexeme, in while_stmt.

    170. for_stmt::=for_clause suite ("else" ":" suite).
    171. for_clause::= "for" target_list "in" expression ":" .
    172. for::lexeme, used for for loops and comprehensions, for_clause, list_for, ...
    173. in::lexeme.

      Semantic note: In for v in e:, e must evaluate to an object that can be iterated over; there are many such objects in Python (including lists, tuples, strings, iterators, generators, and certain user-defined types). A simple expression_list is a tuple display.

      So for i in 20: is syntactically correct but gives a type errors (typically). The following are ok:

       	l=range(3)
       	for i in l:
       	   print i
       	for i in l+['x']:
       	   print i
       	for i in (3*j for j in range(3)):
       	   print i,
       	for i in 1,2,3,4:
       	   print i,
       	for i in [1,2,3,4]:
       	   print i,

      Another complication comes when the target has many variables. This is invalid: for i,j in [6,7]:, but the following is OK

       	for i,j in [1,2],[3,4]:
       	   print i,j
      and outputs
       	1 2
       	3 4

      Note. Earlier grammars the syntax was given as

       "for" $target_list "in" $expression_list ":".

    174. try_stmt::= try1_stmt | try2_stmt.

    175. try1_stmt::= "try" ":" suite N("except" O(expression O("," target)) ":" suite) O("else" ":" suite) O("finally" ":" suite).
    176. try2_stmt::= "try" ":" suite "finally" ":" suite.
    177. try::lexeme, see try_stmt.
    178. except::lexeme, see try_stmt.
    179. else::lexeme, see try_stmt, while_stmt, for_stmt.
    180. finally::lexeme, see try_stmt.

    181. with_stmt::= "with" expression O("as" target) ":" suite.
    182. with::lexeme, used in a with_stmt.

      Function definitions

    183. funcdef::= O(decorators) "def" funcname "(" O(parameter_list) ")" ":" suite.
    184. def::lexeme, defines a function in funcdef.

    185. decorators::= N(decorator).

    186. decorator::= "@" dotted_name O("(" O(argument_list OC) ")") NEWLINE.

    187. dotted_name::= identifier #("." identifier).

    188. parameter_list::= #(defparameter ",") ("*" identifier O("," "**" identifier) | "**" identifier | defparameter OC ).

    189. defparameter::= parameter O("=" expression) .
    190. sublist::= List(parameter) OC.

    191. parameter::= identifier | "(" sublist ")".

    192. funcname::= identifier.

      Class Definitions

    193. classdef::= "class" classname O(inheritance) ":" suite.
    194. class::lexeme, defines a class in a classdef.

      Example of a definition of a simple Counter class with 4 tests:

             1 class Counter:
             2         "Class of counts"
             3         state=0
             4         def _init_(self):
             5                 self.state=0
             6                 print "init"
             7         def count(self):
             8                 self.state=self.state+1
             9                 print "count"
            10         def value(self):
            11                 return self.state
            12
            13 x=Counter()
            14 print x.value()
            15 x.count()
            16 print x.value()
      [A Counter has an integer state and counts...]

    195. inheritance::= "(" O(expression_list) ")".

    196. classname::= identifier.

      Input

    197. file_input::= #(NEWLINE | statement).

    198. interactive_input::= O(stmt_list) NEWLINE | compound_stmt NEWLINE.

    199. eval_input::= expression_list #( NEWLINE).

    200. input_input::= expression_list NEWLINE.

    . . . . . . . . . ( end of section Syntax of the Python Programming Language) <<Contents | End>>

End