UnicodeEncodeError in Python: Causes and Fixes

UnicodeEncodeError happens when Python tries to convert text into bytes, but the target encoding cannot represent one or more characters.

This often appears when:

  • writing text to a file
  • printing text to a terminal
  • sending text to another system
  • calling .encode() with the wrong encoding

For beginners, the fastest fix is usually to use UTF-8 explicitly.

Quick fix

text = "café ☕"

with open("output.txt", "w", encoding="utf-8") as file:
    file.write(text)

Most fixes start by using UTF-8 explicitly. The error often happens when Python uses a narrower encoding like ascii or cp1252.

What this error means

UnicodeEncodeError happens when Python tries to turn a string into bytes using an encoding that cannot represent one or more characters.

A few important ideas:

  • A Python str stores text as Unicode characters.
  • An encoding is the rule used to save or send that text as bytes.
  • Not every encoding supports every character.
  • The error message usually tells you:
    • which encoding Python tried to use
    • which character caused the problem

For example, this text contains characters outside plain ASCII:

  • é
  • ☕

If Python tries to save that text using ascii, it fails because ASCII only supports a small set of characters.

If you need a refresher on text values in Python, see Python strings explained.

When this error usually happens

This error commonly appears in these situations:

  • Writing text to a file without setting encoding
  • Printing text to a terminal that does not support the character
  • Sending text to an API, socket, or external tool with the wrong encoding
  • Calling .encode() with a limited encoding such as ascii

A very common beginner mistake is assuming Python will always pick the right encoding automatically. Sometimes it does, but sometimes it uses the system default instead.

Example that causes the error

Here is a simple example that raises UnicodeEncodeError:

text = "café ☕"
data = text.encode("ascii")
print(data)

Output:

UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 3: ordinal not in range(128)

Why this fails:

  • text is a Python string
  • .encode("ascii") tries to convert it to bytes using ASCII
  • ASCII cannot represent é or ☕
  • Python raises UnicodeEncodeError

Here is the same idea with a character that ASCII can handle:

text = "cafe"
data = text.encode("ascii")
print(data)

Output:

b'cafe'

That works because all characters are valid in ASCII.

How to fix it

Use UTF-8 when opening files

When writing text files, explicitly set the encoding.

text = "café ☕"

with open("output.txt", "w", encoding="utf-8") as file:
    file.write(text)

This is the safest default for most modern Python programs.

If you want to learn more about file handling, see how to write to a file in Python and the Python open() function explained.

Use UTF-8 with encode()

If you need bytes, use UTF-8 unless another system requires something else.

text = "café ☕"
data = text.encode("utf-8")

print(data)

Output:

b'caf\xc3\xa9 \xe2\x98\x95'

This works because UTF-8 can represent these characters.

Choose the correct encoding for the destination

Sometimes you must match another system.

For example, a tool or file format may require a specific encoding such as cp1252 or latin-1. In that case, use the encoding that system expects, but make sure it supports your characters.

text = "café"
data = text.encode("cp1252")
print(data)

This works because cp1252 supports é.

But this fails:

text = "café ☕"
data = text.encode("cp1252")
print(data)

The coffee cup character is not available in cp1252.

Use errors='ignore' or errors='replace' carefully

If you cannot preserve all characters, Python lets you skip or replace unsupported ones.

text = "café ☕"

print(text.encode("ascii", errors="ignore"))
print(text.encode("ascii", errors="replace"))

Output:

b'caf '
b'caf? ?'

Be careful:

  • ignore silently removes characters
  • replace changes characters, usually to ?

These options can be useful, but they may lose data. It is better to use the correct encoding first.

Check terminal or editor encoding

Sometimes the error happens during print() instead of file writing.

text = "café ☕"
print(text)

If this raises UnicodeEncodeError, the problem may be your terminal output encoding, not your string.

In that case:

  • check your terminal settings
  • check Python's stdout encoding
  • try running in a UTF-8 terminal
  • make sure your editor and environment are using UTF-8

How to debug the problem

When you see this error, use a simple process.

1. Read the full error message

Look for:

  • the encoding name, such as ascii
  • the character that failed
  • the position where it happened

That tells you what Python was trying to do.

2. Find where Python is converting text to bytes

Look for code like:

  • text.encode(...)
  • open(..., "w") without encoding=...
  • print(...) in a limited terminal
  • library calls that send text to another system

3. Inspect the text

If the problem is hard to see, print the text with repr().

text = "café ☕"
print(repr(text))

Output:

'café ☕'

This helps you spot hidden characters such as \n, \t, or unexpected Unicode characters.

4. Check file operations

If you are writing a file, review your open() call.

Bad:

with open("output.txt", "w") as file:
    file.write("café ☕")

Better:

with open("output.txt", "w", encoding="utf-8") as file:
    file.write("café ☕")

If you also read text from files, see how to read a file in Python.

5. Check explicit .encode() calls

If you wrote something like this:

text.encode("ascii")

ask yourself:

  • Do I really need ASCII?
  • Should this be UTF-8 instead?
  • Does the other system require a specific encoding?

Safe beginner guidance

For most modern Python code, these rules are safe and practical:

  • Use UTF-8 as your default encoding
  • Do not mix str and bytes unless you understand the difference
  • Keep track of encoding from input to output
  • Avoid using ascii unless you specifically need it

A useful mental model:

  • str = text
  • bytes = raw binary data
  • encoding = turning text into bytes
  • decoding = turning bytes into text

If you want to understand Python errors more generally, see Python errors and exceptions explained.

Common mistakes

These are common causes of UnicodeEncodeError:

  • Using text.encode('ascii') on text that contains accented letters or emoji
  • Writing a file without encoding='utf-8'
  • Running Python in a terminal with a limited default encoding
  • Passing text to a library or tool that expects a different encoding
  • Confusing str objects with bytes objects

Useful commands for debugging

These commands can help you inspect your Python environment.

Check your Python version:

python --version

Check your terminal output encoding:

python -c "import sys; print(sys.stdout.encoding)"

Check your system's preferred encoding:

python -c "import locale; print(locale.getpreferredencoding(False))"

Test a Unicode string and UTF-8 encoding:

python -c "text='café ☕'; print(repr(text)); print(text.encode('utf-8'))"

FAQ

What is the difference between UnicodeEncodeError and UnicodeDecodeError?

UnicodeEncodeError happens when Python turns text into bytes.

UnicodeDecodeError happens when Python turns bytes into text.

If that is your problem instead, see UnicodeDecodeError: utf-8 codec can't decode byte.

Why does this happen only on some computers?

Different systems and terminals may use different default encodings.

One machine may default to UTF-8, while another uses a more limited encoding. That is why the same code may work on one computer and fail on another.

Should I always use UTF-8?

Usually yes.

UTF-8 is the safest default for modern Python code unless you must match a specific external format.

Can I fix this with errors='ignore'?

Yes, but it can silently remove characters.

It is better to use the correct encoding first.

See also