Much of mathematics is written in a peculiar pidgin. If you wish to write in standard English, here are a few things to avoid.

Don't use "so that" when you mean "such that"

Here are the definitions.

so that

  1. In order that, as in I stopped so that you could catch up.
  2. With the result or consequence that, as in Mail the package now so that it will arrive on time.
  3. so ... that. In such a way or to such an extent that, as in The line was so long that I could scarcely find the end of it.
From dictionary.com

such that

  1. adj : of a degree or quality specified (by the "that" clause); their anxiety was such that they could not sleep. (dictionary.com)
  2. A condition used in the definition of a mathematical object. For example, the rationals can be defined as the set of all m/n such that n is nonzero and m and n are integers . (mathworld.wolfram.com)

Examples

  1. We require x to be a rational number so that mx is an integer for some m. [Correct]
  2. We require x to be a rational number so that 3x is an integer. [Incorrect; should be such that 3x is an integer.]
  3. Let H be a discrete subgroup of the Lie group G so that G/H is compact. [Incorrect --- not all discrete subgroups of Lie groups have compact quotient; this is from the Annals of Math., 107, p313.]
  4. Let N and N' be submodules of a module M such that N contains N', so that N/N' is a submodule of M/N'. [Correct! From Steps in Commutative Algebra.]

Briefly, if omitting the "that" from "so that" renders the sentence nonsense, then it was already nonsense, and you should have used "such that". You won't find "so that" among lists of commonly misused expressions because only mathematicians commonly misuse it. Probably the error arose from the influence of German on American (mathematical) English, since the two are not distinguished in German.
[From Matthias Künzer: They are distinguished:
Sei x in Q, so daß es ein m in Z\{0} mit xm in Z gibt.
Sei y in Q so, daß 3y in Z liegt.
Not very many German writers care. "After all, it's just a comma."]

Write "Gowers's Weblog", not "Gowers' Weblog" or "Gower's Weblog" (Gowers)

[Incidentally, The Princeton Companion to Mathematics is written in unusually good English.]

See:

Use an apostrophe plus -s to show the possessive form of a singular noun, even if that singular noun already ends in -s. (grammar.about.com)
Form the possessive singular of nouns by adding 's. (Strunk and White, The Elements of Style, Elementary Rules of Usage, Rule 1.)
The possessive of most singular nouns is formed by adding an apostrophe and an s. (The Chicago Manual of Style, 7.17)
If you write "Gowers' Weblog", not only will you commit a solecism, but you will mislead a literate reader into thinking that you are referring to the weblog of two or more persons called Gower.

For a truly gruesome error, see BAMS 46.4 (2009), p601, line 8, where one finds: "the Langland's program". (Should be "the Langlands program" or "Langlands's program".)

In brief: the rule is add 's (not ') to singular nouns, except that some authorities allow a few exceptions. In my reading I virtually always see 's used except in mathematics.

Don't write "verify" when you mean "satisfy"

"Verify" is often misused by mathematicians for "satisfy", especially by those whose native language is French. For example, Roos (2006) writes: "The reason for this is that AB4* is rarely verified for them." He means satisfied. It is possible for a condition to be always satisfied but rarely verified (for example, the commutativity of a certain class of diagrams).
verify means to prove the truth of; satisfy means to fulfill the requirements of. For example, I believe the function satisfies the Leibniz condition, but I haven't verified [i.e., checked] this.

Don't use "associate to"

Instead use "associate with" or "attach to", whichever is more appropriate. In English, you may associate with gangsters, or attach yourself to the Crips, but you may not associate to either.
Again, only mathematicians seem to get this consistently wrong.

The verbs "permit", "allow", "reduce" normally require objects

For example, the following are incorrect:
The preceding result permits (or allows) to assume that x is positive. [Should be: permits (or allows) us to assume.]
By taking a shortcut, we reduce to 10 miles. [Should be: By taking a shortcut, we reduce the distance to 10 miles.]
We use the local theory to construct the formal group associated to the Neron-model, which then allows to simplify the global steps (Faltings 2008, p93). [Should be: ... formal group attached to... allows us...]

Let now us praise famous men.

Only mathematicians write "Let now" --- real people write "Now let" or "Let us now".

Be careful with quantifiers and negatives

The following statements are equivalent:
  1. a(n) \neq 0 for all n.
  2. a(n) is not equal to zero for all n.
  3. a(n) is not zero for all n.
  4. There exists an n for which a(n) is not zero.
So don't use the first to mean "no a(n) is zero"; instead write "no a(n) is zero" (or at least put a comma before the "for all").

Write sentences that can be parsed by a nonexpert

Serre gives the example of an article that announces its main result in the form:
formula A = formula B = formula C
(no words). Only someone very expert in the field will recognize that the main result is "formula A = formula B" --- the second equality is only an aside.

Don't write "issue" or "challenge" when you mean "problem"

The first is an illiterate euphemism invented by software companies to avoid admitting that their stuff doesn't actually work. The second is feel-good English best left to the New Age crowd.

Don't write "address" when you mean "solve", "examine", "study", or ...

Instead, write "solve", "examine", "study", or whatever it is you actually mean. For example, don't write "the conjecture is successfully addressed"; write "the conjecture is proved" (if that is what you mean).

Be wary of using :=

Personally, I consider the use of "A := B" for "B is defined to be A" or "B equals A by definition" to be ugly programming jargon (especially ugly if the symbols aren't vertically centred--use the package mathtools to fix this), but if you do choose to use it, use it only when the colon genuinely adds something. Note that "We define A:=B" is shorthand for "We define A is defined to be B".