Please, I have an issue on a golang web application. I don’t understand why is happening
Here is the code of the http handler
func (h *HttpServer) GenerateMobileApp(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
pid := vars["id"]
// Check if the project exists : gRPC - to Project micro-services - ask to project-api msvc
// var project *models.Project
// if project, err := h.projectClient.IsProjectExists() isProjectExists(pid); err != nil {
var project *protos.ProjectReply
idRequest := &protos.IDRequest{
Id: pid,
}
project, err := h.projectClient.IsProjectExists(r.Context(), idRequest)
if err != nil {
h.JSON(w, http.StatusInternalServerError, fmt.Errorf("error when checking project existance"))
return
}
if project == nil {
h.JSON(w, http.StatusBadRequest, fmt.Errorf("project does not exists"))
return
}
log.Println("Grpc downloaded Project : \n\t", project)
// Save locally the downloaded project
if err := h.store.SaveDownloadedProject(project); err != nil {
h.JSON(w, http.StatusBadRequest, fmt.Errorf("error when saving the downloaded project"))
return
}
// Download the repository
if err := h.mobileAPP.CloneBoilerplate(project.Id); err != nil {
h.JSON(w, http.StatusBadRequest, fmt.Errorf("error when cloning boilerplate"))
return
}
log.Println("Project successfully clone ", h.mobileAPP)
if err := h.mobileAPP.Update(project); err != nil {
h.JSON(w, http.StatusBadRequest, fmt.Errorf("error when updating boilerplate"))
return
}
log.Println("TODO - Update the project not done yet")
var wg sync.WaitGroup
wg.Add(1)
go func() {
apkGeneratedPath, err := h.mobileAPP.GenerateAPK(project.Id)
if err != nil {
log.Println("error when generation APK")
h.JSON(w, http.StatusBadRequest, fmt.Errorf("error when generating APK"))
wg.Done()
return
}
log.Println("APK Generated correctly: ", apkGeneratedPath)
wg.Done()
}()
wg.Wait()
// Save in the DB the APK generated
h.JSON(w, http.StatusCreated, "link of the play store app")
}
Here is the code of h.mobileAPP.GenerateAPK(project.id)
func (app *MobileAPP) GenerateAPK(projectID string) (string, error) {
log.Println("[GENERATED_APK] Start : ", projectID)
generatedAppFolder := fmt.Sprint("app-", projectID)
generatedAppFolderPath := filepath.Join(CLONED_APP_FOLDER, generatedAppFolder)
os.Chdir(generatedAppFolderPath) // (filepath.Join(CLONED_APP_FOLDER, projectID))
log.Println("[GENERATED_APK] Changed Directory to : ", generatedAppFolderPath)
log.Println("[GENERATED_APK] Running : flutter build appbundle")
cmd := exec.Command("flutter", "build", "appbundle")
stdout, _ := cmd.StdoutPipe()
stderr, _ := cmd.StderrPipe()
cmd.Start()
var wg sync.WaitGroup
wg.Add(1)
go func() {
oneByteStderr := make([]byte, 100)
for {
_, err := stderr.Read(oneByteStderr)
if err != nil {
log.Println("[GENERATED_APK][STDErr] stdout.Read error : ", err.Error())
break
}
r := bufio.NewReader(stderr)
line, _, _ := r.ReadLine()
log.Println("[GENERATED_APK][STDErr] r.ReadLine() ", string(line))
}
wg.Done()
}()
// num := 1
oneByte := make([]byte, 100)
for {
_, err := stdout.Read(oneByte)
if err != nil {
log.Println("[GENERATED_APK] stdout.Read error : ", err.Error())
break
}
r := bufio.NewReader(stdout)
line, _, _ := r.ReadLine()
log.Println("[GENERATED_APK] r.ReadLine() ", string(line))
}
wg.Wait()
err := cmd.Wait()
if err != nil {
log.Fatalf("[GENERATED_APK] cmd.Run() failed with %s\n", err)
}
return "", nil
// // Copy the APK, Rename it
// // Development: app-generated file
// // Production: AWS S3
// apkGeneratedPath := filepath.Join(generatedAppFolderPath, "build/app/outputs/apk/release/app-release.apk")
// apkBackupFolder := filepath.Join(APK_BACKUP_FOLDER, projectID)
// // Check if the backup folder already exists
// log.Println("[GENERATED_APK] Check if the backup folder already exists : ", apkBackupFolder)
// if _, err := os.Stat(apkBackupFolder); os.IsNotExist(err) {
// log.Println("[GENERATED_APK] Backup Folder does not exists. Create it")
// os.MkdirAll(apkBackupFolder, os.ModePerm)
// log.Println("[GENERATED_APK] Backup Folder created: ", apkBackupFolder)
// }
// // Move the APK to the backup folder
// // _, err = copyFile(apkFileGeneratedPath, apkBackupFolder)
// apkGeneratedFinalName := fmt.Sprintf("apk-%s__%s", projectID, time.Now().Format("13-02-2006_15:04:05"))
// apkGeneratedFinalPath := filepath.Join(apkBackupFolder, apkGeneratedFinalName)
// err = os.Rename(apkGeneratedPath, apkGeneratedFinalPath)
// if err != nil {
// log.Println("Error while copying : ", apkGeneratedPath)
// return "", err
// }
// log.Println("[GENERATED_APK] Ends")
// return apkGeneratedFinalPath, nil
}
You can notice that I am running a command flutter build appbundle at cmd := exec.Command("flutter", "build", "appbundle")
The issue I have is that that command doesn’t finish. During it’s execution, after a few minute the http handler restart from the beginning. Not the GeneratedAPK method but the whole handler func (h *HttpServer) GenerateMobileApp restart from the beginning thus causing many flutter build appbundle processes . And it stopped when the first one is done. And it makes the http handler too long.
And in the logs of the corresponding pod, we have
[K8s EVENT: Pod generator-api-6d5b979bb7-wmw7n (ns: default)] Back-off restarting failed container
[K8s EVENT: Pod generator-api-6d5b979bb7-wmw7n (ns: default)] Container image "guitou-app/msvc-generator-api:tilt-3c8bcf3c86172c1e" already present on machine
Detected container restart. Pod: generator-api-6d5b979bb7-wmw7n. Container: generator-api.
How can I detect what is happening? Please, How to solve it?
CodePudding user response：
What is probably happening in your case is, the web-request is taking too long, and the request is canceled - but you have no mechanism in place to cancel your externally executed process(es). So...
When running any webservice always ensure to leverage context.Context for the lifetime of a request - especially when running any potentially blocking operations (as you are with exec.Command).
So first, from your handler, grab the request's context.Context, and pass this along to any potentially blocking calls:
func (h *HttpServer) GenerateMobileApp(w http.ResponseWriter, r *http.Request) {
...
ctx := r.Context() // context.Context for the lifetime of the request
...
go func() {
apkGeneratedPath, err := h.mobileAPP.GenerateAPK(ctx, project.Id) // <- add ctx
}
}
and update your APK generator signature like so:
func (app *MobileAPP) GenerateAPK(ctx context.Context, projectID string) (string, error)
and to use ctx when executing an external tool:
//cmd := exec.Command("flutter", "build", "appbundle")
cmd := exec.CommandContext(ctx, "flutter", "build", "appbundle")
with this in place, if the web request is canceled (times out - based on web server settings; or the client disconnects) the process will be terminated.
You can even have special clean-up logic in place for this occurrence, by checking the error from exec.CommandContext for either context.Canceled (client disconnected) or context.DeadlineExceeded (request timed-out).
CodePudding user response：
To determine the place where the program hangs up add to your code:
func dumpStackAfter(interval time.Duration) {
go func() {
time.Sleep(interval)
pprof.Lookup("goroutine").WriteTo(os.Stdout, 1)
panic("timeouted")
}()
}
And call it at the beginning of main function:
dumpStackAfter(time.Second*30)
Run you program and in 30 sec it will print stack trace which will show where the program hanged up.