Dulce Pham

Dulce Pham: Does The Conversation Feel Human?

This is how I evaluate the naturalness of the chat with the model. This includes the ability to respond to shifts in tone, contextual awareness, a non-scripted pacing, and consistency to the persona across prompt types.

Primarily I’m looking for an engaging chat that feels non-scripted.

Mixed Signals & Personality Consistency

I began the conversation in a somewhat cheeky and indecisive way. I stopped mid-sentence and almost seemed to challenge the model to adjust its tone on a dime.

I said “Wait… No go on.” I did this on purpose to see if it would understand the conflicting commands. When I asked it what it was thinking about the model replied in a brash and sassy way.

It committed to its personality and came across as cheeky and somewhat menacing. This told me that it was doing a good job of staying within the personality bounds.

Tone Shift & De-Escalation

I am intentionally shifting tone here. I am no longer cheeky and ambiguous. I am trying to bring things down a notch and get into a more serious conversation.

I literally told the model that I was no longer interested in being cheeky and that I wanted a more serious discussion. This was a test to see if it could emotionally pivot and follow the cues.

The model replied in character but was more serious and cautious in its response. I could tell it was attempting to understand me a bit more. This told me that it was able to pivot tone without blowing up the character.

Low-Pressure Prompt & Strong Character Retention

Here I am coming at the model with a much more relaxed and casual prompt. I simply ask the model if it has time and what its vibe is. I was looking to see how it would respond to a casual prompt.

Instead of responding with a casual tone, the model dug deeper into the dramatic character it had established. It came across as over the top and melodramatic.

It was clear at this point that it was going to hold onto the personality at all costs. This is good for character retention but may pose a challenge if you are building a more free-flowing conversational model.

Visual Output Evaluation

This is a test of consistency of the model’s appearance in generated images. I’m looking at the consistency of appearance (across pose and lighting scenarios), facial lighting, and photorealism of the images.

I’m also evaluating whether the model appears to be the same person in the generated images.

I zoom in to test facial detail and expression.

I want to see how the AI performs in close-up, particularly when it comes to facial texture, skin, and symmetry, as well as the emotional expression of the person.

He looks smiling and yet, still very serious. This seems to keep in line with the rest of the results I have been getting, which suggests that the model has retained the original identity.

Testing full-body character consistency 1/2

I tested how the AI could generate a full body image that retains the character’s identity and style. I wanted to test how it would go when it comes to clothes, style, posture, and general attitude in a classroom.

The character is confidently posing with a very photogenic and professional looking posture, which seems to suggest that the model is heavily focused on the character and the style.

AI Video Generation Review

This is the test of motion. I’m looking for photorealism and smoothness of motion (even if it’s very subtle), consistency from frame-to-frame, and photorealism of the output, does this feel like video or does it feel like a series of images stitched together.

Testing Motion Consistency and Character Presence

I was testing to see if I could get the AI to generate frames with the character moving. The AI-generated character is shown standing in a classroom, making minor movements and looking directly at the camera.

I was testing to see if I could get the AI to generate frames in which the character’s clothes, face, and general aura remained roughly the same.

She moves slightly in the frames shown above, but the motion isn’t very dynamic, which suggests to me that the model is favouring stability over trying to generate interesting motion.