OpenCV
to Grasshopper
Hello people!
In this article, I will show you how to use face detection in OpenCV/python and send the required data to Grasshopper Rhino for design application. OpenCV is one of the most powerful and widely used libraries in both C++ and python. It stands for open-source computer vision. In short, it is a library used for Image Processing. It is mainly used to do all the operation related to images, and of course, videos too.
In the following example, I apply the face detection technics in python and sending the required data to the Grasshopper through the user datagram protocol(UDP). Please check on the following video!
Following paragraph, we will cover,
- why sending data from Python IDE to Grasshopper
- a quick introduction about OpenCV and face detection
- how to send data from Python IDE to Rhino Grasshopper by User Datagram Protocol (UDP)
- a script in grasshopper to visualize the result
Why sending the data from python IDE* to Grasshopper?
You may be wondering why we need to do this? Indeed, there is the default python editor(plugin) in grasshopper. However, It does not support the OpenCV library and also other frequent used libraries in python. Therefore, cross-platform data transfer will be useful in solving this issue.
The reason why the default python plugin doesn’t support lots of libraries is that it is a different framework of python that we familiar. The existing python in grasshopper is what we call IronPython. I know it sounds a little bit confusing, but let me explain.
Python is a language and also implementation. Language implementation is a system for executing computer programs. While we say “IronPython” or “CPython,” they are language implementations, not the language itself. The name “CPython” means that it is the implementation of Python in C language. To find out more about it, I will recommend you read this.
*IDE stands for Integrated Development Environment
About the OpenCV and Face detection technics
what is OpenCV?
Again, OpenCV stands for the open-source computer vision. It a library in python and It can help us in real-time computer vision and also image processing. Therefore, if you are interested in robotic fabrication in architecture, OpenCV might be something that you will want to pick up. Moreover, the application of the computer vision combining machine learning makes the tool more powerful. It allows us to do all kinds of object detection and recognition.
For more theory and code, I will write another post related to the topic. For now, you might want to refer to here. has well explained the idea behind the scene. For the code that I use in the example, you can download from here.
how to send data from Python IDE to Rhino Grasshopper via User Datagram Protocol (UDP)
The first question in your mind might be, “What the hell is UDP? I know Bluetooth and airdrop maybe.”
In short, UDP is a communication/data transfer protocol based on your Internet protocol (IP address). Also, there is another transfer protocol, Transmission Control Protocol (TCP). The main difference is that TCP is connection-oriented, but UDP is connection-less. This means that TCP tracks all data sent, requiring an acknowledgment for each byte. UDP does not use acknowledgments at all. There are pro and cons for both protocols. For more comparison, please check here.
The reason why I decided to go with UDP is that I am not interested in the acknowledgment process. The back and forward process are redundant in this case.
In this client-server model, python IDE will be the server to provide the data, and the Grasshopper will be the client that receive the data. In Python, we can use the built-in library socket to achieve it.
import socket
def UDP (IP, port,message):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(message, "utf-8"),(IP,port))
In this four-line code, first, we import the socket library, and we define a function that requires three inputs. An IP address that you want to send to, the port that you are using to send and finally, the message that you want to send.
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
Where socket.AF_INET represent the IPv4 DNS server and the socket.SOCK_DGRAM means the UDP protocol that we want to use.
sock.sendto(bytes(message, "utf-8"),(IP,port))
If you download the code from my Github, then you will notice that the message that I sent is a string. However, the socket library only accepts the “byte-like” object; therefore, I need to transfer the string to the byte.
If you are still following, we successfully send the data to the assigned IP address from Python IDE via the port. We need someone else to pick up the package! There are several options to do this. In this case, I used the plug-in from ghowl, which is developed by Damien Alomar, and Luis E. Fraguada and Giulio Piacentino. Open the script, “opencv2gh.gh” from the GitHub package. You will see there are three inputs from the component “UDP Receive.” With assigning the IP and port number, you can receive the real-time data from the python now!
Visualize the data in Grasshopper
About the data I send to Grasshopper include the center of the face and the size of the face. you may refer to the red line and the blue dot in the following image.
Rest of the script is just trying to manipulate the data to apply in the geometry transformation. In this example, I simply have the human face as the attractor to control the opening of the facade. Of course, there are thoundsands of design approach that you can try with. Imagination and creativity is your super power!
Finally, let’s have a quick recap of what happened in this article.
- why we did not use the default python in Grasshopper
- basis introduction for OpenCV library and face detection technics
- what are UDP and TCP
- How to send data via internet protocol
- data application in design
That will be all for this post, thanks for reading and hope you enjoy it!
