node.js - Parsing error when sending data from godot to api - Stack Overflow

I am using Godot version 4.1.4 to make an application. In it, I have some stories, and each page of each story contains text and an image. I am trying to use a RESTful API (Node.js) to add the data to my MySQL database. The code in my API works, and I am able to send over the needed data using Postman. I am using multipost forms to send the data, with a user_id, title, pages, and image fields. It should store a URL to the image in the database, and then the actual image in a directory in my server. In my godot project, I am not sure how to send over the Image. I am currently using FileAccess to open the file, then I use get_as_text()to get the image data as text, and I add this data to my boundary. currently, when I run the program, it gives me a parsing error: "Unicode parsing error, some characters were replaced with � (U+FFFD): Invalid UTF-8 leading byte (89)". Is there a way to fix this, or a better way to send the images? Any help is appreciated. Here is the godot code I am using:

func upload_story():
    var url = "http://localhost:3000/add-story"
    # Prepare HTTPRequest
    var http_request = HTTPRequest.new()
    add_child(http_request)
    http_request.request_completed.connect(self._on_request_completed)
    #replace with multi image
    var file = FileAccess.open("res://Assets/images/Story Images/Story1/student.png", FileAccess.READ)
    var file_data = file.get_as_text()
    var form_data = {
        "user_id": "1",
        "title": "MyStoryExample",
        "pages": [
    {"page_number": 1, "text_content": "It was a sunny day at the park."},
    {"page_number": 2, "text_content": "Children were playing, and birds chirped."}
    ],
        "images": file_data  # This is the image data as bytes
    }
    var boundary = "----WebKitFormBoundary7MA4YWxkTrZu0gW"
    var body = _generate_multipart_data(form_data, boundary)
    var headers = [
        "Content-Type: multipart/form-data; boundary=" + boundary
    ]
    http_request.request(url, headers, HTTPClient.METHOD_POST, body)




func _on_request_completed(result, response_code, headers, body):
    if response_code == 201:
        print("Request Succeded")
        print(str(body))
    else:
        print("Request Failed with response code: ", response_code)



func _generate_multipart_data(data, boundary):
    var body = ""
    for key in data.keys():
        var value = data[key]
        # For image data, we need to handle the file part specifically
        if key == "images":
            body += "--" + boundary + "\r\n"
            body += "Content-Disposition: form-data; name=\"" + key + "\"; filename=\"image.png\"\r\n"
            body += "Content-Type: image/png\r\n\r\n"
            body += value
            body += "\r\n"
        else:
            body += "--" + boundary + "\r\n"
            body += "Content-Disposition: form-data; name=\"" + key + "\"\r\n\r\n"
            body += str(value) + "\r\n"
    body += "--" + boundary + "--\r\n"
    return body

I have tried using base64 encoding, which did not work as I wanted it to and would make the process much harder later on. I've tried using utf_8 encoding and getting the data in other ways, but it has not worked for me yet, and i'm not sure where my errors are coming from.

I am using Godot version 4.1.4 to make an application. In it, I have some stories, and each page of each story contains text and an image. I am trying to use a RESTful API (Node.js) to add the data to my MySQL database. The code in my API works, and I am able to send over the needed data using Postman. I am using multipost forms to send the data, with a user_id, title, pages, and image fields. It should store a URL to the image in the database, and then the actual image in a directory in my server. In my godot project, I am not sure how to send over the Image. I am currently using FileAccess to open the file, then I use get_as_text()to get the image data as text, and I add this data to my boundary. currently, when I run the program, it gives me a parsing error: "Unicode parsing error, some characters were replaced with � (U+FFFD): Invalid UTF-8 leading byte (89)". Is there a way to fix this, or a better way to send the images? Any help is appreciated. Here is the godot code I am using:

func upload_story():
    var url = "http://localhost:3000/add-story"
    # Prepare HTTPRequest
    var http_request = HTTPRequest.new()
    add_child(http_request)
    http_request.request_completed.connect(self._on_request_completed)
    #replace with multi image
    var file = FileAccess.open("res://Assets/images/Story Images/Story1/student.png", FileAccess.READ)
    var file_data = file.get_as_text()
    var form_data = {
        "user_id": "1",
        "title": "MyStoryExample",
        "pages": [
    {"page_number": 1, "text_content": "It was a sunny day at the park."},
    {"page_number": 2, "text_content": "Children were playing, and birds chirped."}
    ],
        "images": file_data  # This is the image data as bytes
    }
    var boundary = "----WebKitFormBoundary7MA4YWxkTrZu0gW"
    var body = _generate_multipart_data(form_data, boundary)
    var headers = [
        "Content-Type: multipart/form-data; boundary=" + boundary
    ]
    http_request.request(url, headers, HTTPClient.METHOD_POST, body)




func _on_request_completed(result, response_code, headers, body):
    if response_code == 201:
        print("Request Succeded")
        print(str(body))
    else:
        print("Request Failed with response code: ", response_code)



func _generate_multipart_data(data, boundary):
    var body = ""
    for key in data.keys():
        var value = data[key]
        # For image data, we need to handle the file part specifically
        if key == "images":
            body += "--" + boundary + "\r\n"
            body += "Content-Disposition: form-data; name=\"" + key + "\"; filename=\"image.png\"\r\n"
            body += "Content-Type: image/png\r\n\r\n"
            body += value
            body += "\r\n"
        else:
            body += "--" + boundary + "\r\n"
            body += "Content-Disposition: form-data; name=\"" + key + "\"\r\n\r\n"
            body += str(value) + "\r\n"
    body += "--" + boundary + "--\r\n"
    return body

I have tried using base64 encoding, which did not work as I wanted it to and would make the process much harder later on. I've tried using utf_8 encoding and getting the data in other ways, but it has not worked for me yet, and i'm not sure where my errors are coming from.

• node.js
• godot
• godot4

Share Improve this question Follow asked 13 hours ago chase pelechytikchase pelechytik 122 bronze badges New contributor chase pelechytik is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. 1

• Why did base64 not work for you? Where did you run into problems there? If you use get_file_as_bytes and Marshalls.raw_to_base64() you shouldn't run in any encoding problems. – Bugfish Commented 6 hours ago

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

You should use PackedByteArray to pack the body data and pass it to HttpRequest's request_raw().

Use the correct string buffer method: utf8, ascii, what your server expects.

And with the correct Content-Type append your image as a byte buffer.

For example:

var line = ""
var body = PackedByteArray()

line = "Content-Disposition: form-data; name=\"" + key + "\"; filename=\"image.png\"\r\n"
body.append_array(line.to_utf8_buffer())

line = "Content-Type: image/png\r\n\r\n"
body.append_array(line.to_utf8_buffer())

var img_bytes = FileAccess.get_file_as_bytes("image.png")
body.append_array(img_bytes)

# ...

# Then call request_raw()

This is wholly incomplete. Hope it helps.